Vehicle monitoring system and vehicle monitoring method

ABSTRACT

A vehicle monitoring system includes at least one camera, and a server that is communicably connected to a client terminal. The camera performs capturing a vehicle while switching a first capturing condition including an image parameter for capturing a number of the vehicle entering an angle of view of the camera and a second capturing condition including an image parameter for capturing an occupant&#39;s face of the vehicle, and transmits a first captured video under the first capturing condition and a second captured video under the second capturing condition to the server. The server arranges reproduction screens for the first captured video and the second captured video that are reproduceable in the client terminal and displays the reproduction screens on the client terminal based on the first captured video and the second captured video.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.16/875,412, filed on May 15, 2020, which in turn claims the benefit ofJapanese Application No. 2019-094824, filed on May 20, 2019. Thedisclosure of each of these documents, including the specification,drawings, and claims, is incorporated herein by reference in itsentirety.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to a vehicle monitoring system and avehicle monitoring method capable of imaging and monitoring a vehicle asa subject.

2. Background Art

There is a technique in which a plurality of cameras are disposed at apredetermined location on a traveling path of a vehicle, and cameraimage information captured by each camera is displayed on a displaydevice of a terminal apparatus mounted on the vehicle via a network anda wireless information exchanging apparatus (for example, refer toJP-A-2007-174016). According to JP-A-2007-174016, a user can obtainreal-time camera images with a large information amount on the basis ofthe camera image information captured by the plurality of camerasdisposed on the traveling path of the vehicle.

SUMMARY OF THE INVENTION

In JP-A-2007-174016, in a case where a vehicle as a subject is imaged tobe used for monitoring, it is not taken into consideration that thevehicle is imaged while switching different imaging conditions such thata monitoring person (for example, a police officer) can recognize uniquefeatures (for example, a face of an in-vehicle occupant such as a driverdriving the vehicle and a number plate of the vehicle) of the travelingvehicle, and captured moving images are displayed in a contrastedmanner. For example, in a case where a case or the like occurs, asuspect who has caused the case or the like may ride on a vehicle andescape, and thus it is necessary to identify a face of an occupant and anumber plate of a runaway vehicle in an early stage in policeinvestigation. However, even though the technique in JP-A-2007-174016 isused, when an investigator (for example, a police officer) checks acaptured video of each camera one by one, there is a problem in that theinvestigation takes time, which is not efficient, and it is difficult todetect a suspect early.

The present disclosure has been made in light of the circumstances ofthe related art, and is directed to providing a vehicle monitoringsystem and a vehicle monitoring method capable of assisting a policeofficer in identifying unique features of a runaway vehicle on which asuspect having caused a case or the like is riding in an early stage andvisually, and thus improving convenience of police investigation.

According to the present disclosure, there is provided a vehiclemonitoring system including at least one camera; and a server that iscommunicably connected to a client terminal, in which the cameraperforms capturing a vehicle while switching a first capturing conditionincluding an image parameter for capturing a number of the vehicleentering an angle of view of the camera and a second capturing conditionincluding an image parameter for capturing an occupant's face of thevehicle, and transmits, to the server, a first captured video under thefirst capturing condition and a second captured video under the secondcapturing condition, and in which the server arranges reproductionscreens for the first captured video and the second captured video thatare reproduceable in the client terminal and displays the reproductionscreens on the client terminal based on the first captured video and thesecond captured video.

According to the present disclosure, there is provided a vehiclemonitoring method performed by a vehicle monitoring system including atleast one camera, and a server that is communicably connected to aclient terminal, the vehicle monitoring method including by the camera,performing capturing a vehicle while switching a first capturingcondition including an image parameter for capturing a number of thevehicle entering an angle of view of the camera and a second capturingcondition including an image parameter for capturing an occupant's faceof the vehicle, and transmit, to the server, a first captured videounder the first capturing condition and a second captured video underthe second capturing condition; and by the server, arrangingreproduction screens for the first captured video and the secondcaptured video that are reproduceable in the client terminal and displaythe reproduction screens on the client terminal based on the firstcaptured video and the second captured video.

According to the present disclosure, it is possible to identify uniquefeatures of a runaway vehicle on which a suspect having caused a case orthe like is riding with high efficiency and thus to improve convenienceof police investigation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram illustrating a first example of a systemconfiguration of a vehicle monitoring system according to Embodiment 1.

FIG. 1B is a diagram illustrating a second example of a systemconfiguration of a vehicle monitoring system according to Embodiment 1.

FIG. 2 is a block diagram illustrating a hardware configuration exampleof a vehicle imaging camera in detail.

FIG. 3A is a flowchart illustrating an operation procedure examplerelated to imaging performed by the vehicle imaging camera.

FIG. 3B is a flowchart illustrating an operation procedure examplerelated to data transmission performed by the vehicle imaging camera.

FIG. 3C is a flowchart illustrating an operation procedure examplerelated to lighting control performed by the vehicle imaging camera.

FIG. 3D is a table illustrating various operation control examples ineach of a daytime mode and a nighttime mode.

FIG. 4 is a block diagram illustrating a hardware configuration exampleof a video accumulation/analysis server in detail.

FIG. 5A is a sequence diagram illustrating a first example of anoperation procedure regarding video analysis in the vehicle monitoringsystem.

FIG. 5B is a sequence diagram illustrating a second example of anoperation procedure regarding video analysis in the vehicle monitoringsystem.

FIG. 5C is a sequence diagram illustrating a third example of anoperation procedure regarding video analysis in the vehicle monitoringsystem.

FIG. 5D is a sequence diagram illustrating a fourth example of anoperation procedure regarding video analysis in the vehicle monitoringsystem.

FIG. 6A is a flowchart illustrating a first example of an operationprocedure of a process in which the vehicle imaging camera registers adriver and a passenger.

FIG. 6B is a flowchart illustrating a second example of an operationprocedure of a process in which the vehicle imaging camera registers adriver and a passenger.

FIG. 7 is a diagram illustrating a monitoring screen example of acaptured video in the daytime mode.

FIG. 8 is a diagram illustrating an example of a monitoring screendisplaying a moving image reproduction screen in the daytime mode onwhich an in-vehicle occupant is clearly reflected and a moving imagereproduction screen in the daytime mode on which a number plate isclearly reflected.

FIG. 9 is a diagram illustrating an example of a monitoring screendisplaying a moving image reproduction screen in the nighttime mode onwhich an occupant in a vehicle is clearly reflected and a moving imagereproduction screen in the nighttime mode on which a number plate isclearly reflected.

FIG. 10 is a diagram illustrating an example of a monitoring screendisplaying a retrieval result of a face image of an in-vehicle occupantusing a number plate that is input as a retrieval condition.

FIG. 11A is a diagram illustrating a first example of a retrieval resultscreen displayed subsequently to the monitoring screen in FIG. 10 .

FIG. 11B is a diagram illustrating a second example of a retrievalresult screen displayed subsequently to the monitoring screen in FIG. 10.

FIG. 12 is a diagram illustrating a third example of a retrieval resultscreen displayed subsequently to the monitoring screen in FIG. 10 .

FIG. 13 is a sequence diagram illustrating an example of a series ofoperation procedures of retrieving a face image of an in-vehicleoccupant by using a number plate that is input as a retrieval condition.

FIG. 14 is a diagram illustrating an example of a retrieval resultscreen displaying a retrieval result of a number plate that is retrievedby using a face image of an in-vehicle occupant that is input as aretrieval condition.

FIG. 15 is a diagram illustrating a first example of a retrieval resultscreen displayed subsequently to the retrieval result screen in FIG. 14.

FIG. 16A is a diagram illustrating a second example of a retrievalresult screen displayed subsequently to the retrieval result screen inFIG. 14 .

FIG. 16B is a diagram illustrating a third example of a retrieval resultscreen displayed subsequently to the retrieval result screen in FIG. 14.

FIG. 17 is a sequence diagram illustrating an example of a series ofoperation procedures of retrieving a number plate by using a face imageof an in-vehicle occupant that is input as a retrieval condition.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENT Details Leading toPresent Disclosure

In JP-A-2007-174016, since camera image information captured by each ofa plurality of cameras is displayed on the display device of theterminal apparatus mounted on the vehicle, a user (for example, adriver) can check the camera image information in real time at alocation where each camera is disposed. However, in JP-A-2007-174016, itis not taken into consideration that, in a case where a vehicle as asubject is imaged to be used for monitoring, an image captured by thecamera is subjected to image processing such that unique features (forexample, a face of an in-vehicle occupant such as a driver driving thevehicle and a number plate of the vehicle) of the traveling vehicle canbe specifically identified. For example, in a case where a case or thelike occurs, a suspect who has caused the case or the like may ride avehicle and escape, it is necessary to efficiently narrow a runawayvehicle in police investigation. However, even though the technique inJP-A-2007-174016 is used, when an investigator (for example, a policeofficer) checks a captured video of each camera one by one, there is aproblem in that the investigation takes time, which is not efficient,and it is difficult to detect a suspect early.

Therefore, in the following embodiment, a description will be made of avehicle monitoring system and a vehicle monitoring method capable ofefficiently identify unique features of a runaway vehicle on which asuspect having caused a case or the like is riding and thus improvingconvenience of police investigation.

Meanwhile, in JP-A-2007-174016, in a case where a vehicle as a subjectis imaged to be used for monitoring, it is not taken into considerationthat the vehicle is imaged while switching different imaging conditionssuch that a monitoring person (for example, a police officer) canrecognize unique features (for example, a face of an in-vehicle occupantsuch as a driver driving the vehicle and a number plate of the vehicle)of the traveling vehicle, and captured moving images are displayed in acontrasted manner. For example, in a case where a case or the likeoccurs, a suspect who has caused the case or the like may ride on avehicle and escape, and thus it is necessary to identify a face of anoccupant and a number plate of a runaway vehicle in an early stage inpolice investigation. However, even though the technique inJP-A-2007-174016 is used, when an investigator (for example, a policeofficer) checks a captured video of each camera one by one, there is aproblem in that the investigation takes time, which is not efficient,and it is difficult to detect a suspect early.

Therefore, in the following embodiment, a description will be made of avehicle monitoring system and a vehicle monitoring method capable ofassisting a police officer in identifying unique features of a runawayvehicle on which a suspect having caused a case or the like is riding inan early stage and visually, and thus improving convenience of policeinvestigation.

Vehicle Monitoring System According to Present Disclosure

Hereinafter, with reference to the accompanying drawings as appropriate,an embodiment in which a vehicle monitoring system and a vehiclemonitoring method according to the present disclosure are specificallydisclosed will be described in detail. However, a detailed descriptionmore than necessary may be omitted. For example, a detailed descriptionof a well-known content or a repeated description of the substantiallysame configuration will be omitted. This is to prevent the followingdescription from being unnecessarily redundant and to facilitateunderstanding of a person skilled in the art. The accompanying drawingsand the following description are provided to enable a person skilled inthe art to fully understand the present disclosure and are not intendedto limit the subject matter disclosed in the claims.

Embodiment 1

FIG. 1A is a diagram illustrating a first example of a systemconfiguration of a vehicle monitoring system 1 according toEmbodiment 1. FIG. 1B is a diagram illustrating a second example of asystem configuration of a vehicle monitoring system 1A according toEmbodiment 1. In description of the vehicle monitoring system 1A in FIG.1B, the same constituent element as that of the vehicle monitoringsystem 1 in FIG. 1A will be given the same reference numeral,description thereof will be simplified or omitted, and differentcontents will be described.

The vehicle monitoring system 1 is configured to include a plurality ofvehicle imaging cameras 10, 10A, 10B, . . . , and a videoaccumulation/analysis server 50. The vehicle monitoring system 1 may beconfigured to further include a client terminal 90. The vehicle imagingcameras 10, 10A, 10B, . . . , and the video accumulation/analysis server50 are communicably connected to each other via a network NW1 such as acommunication line of an intranet. The network NW1 is configured with awired communication line (for example, an optical communication networkusing optical fibers), but may be configured with a wirelesscommunication network. The video accumulation/analysis server 50 may bebuilt as an on-premise server in a police station, and may be providedas a cloud server connected to a network such as the Internet.

Each of the video accumulation/analysis server 50 and the clientterminal 90 is illustrated alone, but each thereof may be provided in aplurality. The vehicle monitoring system 1 may not be used for only asingle police station, and may be applied to an example of cooperativeinvestigation among a plurality of police stations.

Each of the vehicle imaging cameras 10, 10A, 10B, . . . as an example ofa camera is provided in a stationary manner on a pole (not illustrated)erected from a corner of an intersection for each intersection, has apredetermined angle of view, and images a vehicle VCL1 (an example of asubject) entering the intersection. Therefore, intersections where therespective vehicle imaging cameras 10, 10A, 10B, . . . are provided aredifferent from each other. The respective vehicle imaging cameras may beprovided not at all intersections but at some intersections, or may beprovided at a location (a major arterial road such as a national road ora prefectural road) other than an intersection. Each of the vehicleimaging cameras 10, 10A, 10B, . . . sends a captured video to the videoaccumulation/analysis server 50 via the network NW1. In the followingdescription, the captured video includes not only captured video databut also information regarding a camera identification (ID) (in otherwords, position information of the intersection where the camera isprovided) of the camera having generated the captured video and theimaging date and time. Each of the vehicle imaging cameras 10, 10A, 10B,. . . performs imaging while switching a first imaging conditionincluding an image parameter (which will be described later) appropriatefor imaging a number plate NPL1 of the vehicle VCL1 and a second imagingcondition including an image parameter (which will be described later)appropriate for imaging a face FCE1 of an occupant of the vehicle VCL1.

Internal configurations of the vehicle imaging cameras 10, 10A, 10B, . .. may be the same as each other except a configuration of a lens block17 (refer to FIG. 2 ) or a configuration of a camera casing. Forexample, the vehicle imaging camera 10 has a monocular lensconfiguration, and generates two video streams such as a captured video(an example of a first captured video) appropriate for imaging a numberplate and a captured video (an example of a second captured video)appropriate for imaging a face of an occupant. The vehicle imagingcamera 10A is configured with two cameras each having a monocularcamera, and each camera generates a captured video (an example of afirst captured video) appropriate for imaging a number plate and acaptured video (an example of a second captured video) appropriate forimaging a face of an occupant. The vehicle imaging camera 10B has abinocular lens configuration, and generates a captured video (an exampleof a first captured video) appropriate for imaging a number plate and acaptured video (an example of a second captured video) appropriate forimaging a face of an occupant. Thus, in the following description, aconfiguration of the vehicle imaging camera 10 will be described as anexample.

The video accumulation/analysis server 50 as an example of a server isprovided in a police station, and is configured to include, for example,at least a number detection/collation core 51, a vehicle type/vehiclecolor detection core 52, a face detection/collation core 53, and anumber-of-occupants measurement core 54 (refer to FIG. 4 ). The videoaccumulation/analysis server 50 receives and accumulates captured videossent from the respective vehicle imaging cameras 10, 10A, 10B, . . .provided at installation locations (for example, intersections) in ajurisdiction area of the police station, and executes an analysisprocess (for example, refer to FIGS. 5A, 5B, 5C, and 5D) which will bedescribed later. The video accumulation/analysis server 50 provided inthe police station and the client terminal 90 are communicably connectedto each other via a network (not illustrated) such as an intranet in thepolice station.

The number detection/collation core 51 is, for example, a processingunit realized by a processor (which will be described later) configuringthe video accumulation/analysis server 50. The numberdetection/collation core 51 performs a process of detecting a numberplate of a vehicle reflected in captured videos sent from the respectivevehicle imaging cameras 10, 10A, 10B, . . . , or performs a process ofcollating numbers with each other by using a number collation list (notillustrated) preserved in the video accumulation/analysis server 50, ata predetermined timing (for example, refer to FIGS. 5A to 5D). Thenumber detection/collation core 51 stores a processing result of thedetection process or the collation process in a database 57 (refer toFIG. 4 ). The processing result may include, for example, not only anumber detection result or a number collation result but also theimaging date and time for a captured video that is a target of thedetection process or the collation process and a camera ID.

The vehicle type/vehicle color detection core 52 is, for example, aprocessing unit realized by a processor (which will be described later)configuring the video accumulation/analysis server 50. The vehicletype/vehicle color detection core 52 performs a process (analysisprocess) of detecting a vehicle type (for example, a sedan, a wagon, abus, or a truck) and a vehicle color (for example, red, blue, white, orblack) of a vehicle reflected in captured videos sent from therespective vehicle imaging cameras 10, 10A, 10B, . . . at apredetermined timing (for example, refer to FIGS. 5A to 5D). The vehicletype/vehicle color detection core 52 stores a processing result(analysis result) of the detection process in the database 57 (refer toFIG. 4 ). The processing result may include, for example, not onlyanalysis results of a vehicle type and a vehicle color but also theimaging date and time for a captured video that is a target of thedetection process or the collation process and a camera ID.

The face detection/collation core 53 is, for example, a processing unitrealized by a processor (which will be described later) configuring thevideo accumulation/analysis server 50. The face detection/collation core53 performs a process of detecting a face (for example, a driver on adriver seat or a passenger on a passenger seat) of an in-vehicleoccupant of a vehicle reflected in captured videos sent from therespective vehicle imaging cameras 10, 10A, 10B, . . . , or performs aprocess of collating faces with each other by using a face collationlist (for example, a blacklist in which face images of ex-convicts areregistered; not illustrated) preserved in the videoaccumulation/analysis server 50, at a predetermined timing (for example,refer to FIGS. 5A to 5D). The face detection/collation core 53 stores aprocessing result of the detection process or the collation process in adatabase 57 (refer to FIG. 4 ). The processing result may include, forexample, not only a face detection result or a face collation result butalso the imaging date and time for a captured video that is a target ofthe detection process or the collation process and a camera ID.

The number-of-occupants measurement core 54 is, for example, aprocessing unit realized by a processor (which will be described later)configuring the video accumulation/analysis server 50. Thenumber-of-occupants measurement core 54 performs a process (analysisprocess) of measuring the number of in-vehicle occupants of a vehiclereflected in captured videos sent from the respective vehicle imagingcameras 10, 10A, 10B, . . . at a predetermined timing (for example,refer to FIGS. 5A to 5D). The number-of-occupants measurement core 54stores a processing result (analysis result) of the measurement processin the database 57 (refer to FIG. 4 ). The processing result mayinclude, for example, not only analysis results of the number ofoccupants but also the imaging date and time for a captured video thatis a target of the detection process or the collation process and acamera ID.

The client terminal 90, which is provided in the police station, is usedby an operator OP1 (a police officer in the station) in the policestation, and is configured by using a laptop or desktop personalcomputer (PC). For example, in a case where a case or the like occurs,the operator OP1 listens to various pieces of witness informationregarding the case through an incoming call from a reporter (forexample, a witness) who has reported the occurrence of the case or thelike to the police station, and operates the client terminal 90 to inputand record data. The client terminal 90 is not limited to the PC, andmay be a computer having a communication function, such as a smartphoneor a tablet terminal. The client terminal 90 sends, for example, aretrieval request for causing the video accumulation/analysis server 50to retrieve (for example, refer to FIG. 13 or 17 ) a vehicle (that is, arunaway vehicle on which a suspect of the case or the like is riding)matching witness information to the video accumulation/analysis server50, and receives a retrieval result and displays the retrieval result ona display unit 93.

The client terminal 90 is configured to include a communication unit 91,a memory 92, the display unit 93, and a processor PRC2. Although notillustrated in FIGS. 1A and 1B, an operation unit (for example, a mouseor a keyboard) receiving an operation from the operator OP1 may beconnected to the client terminal 90. A user interface (UI) operationsignal from the operation unit is input to the processor PRC2 of theclient terminal 90 due to an operation of the operator OP1. For example,in a case where the operator OP1 listens to the date and time and alocation where a case or the like occurred, and a number through anincoming call from a witness, a UI operation signal for designating thedate and time, the location, and the number as retrieval conditions isinput to the client terminal 90 on the basis of an operation of theoperator OP1.

The communication unit 91 performs wired or wireless communication withthe video accumulation/analysis server 50 connected thereto via anetwork such as an intranet.

The memory 92 is configured by using, for example, a random accessmemory (RAM) and a read only memory (ROM), and temporarily stores aprogram required to execute an operation of the client terminal 90, anddata or information generated during an operation thereof. The RAM is,for example, a work memory used during an operation of the processorPRC2. The ROM stores in advance, for example, a program for controllingthe processor PRC2. The memory 92 may include, for example, a hard diskdrive or a solid state drive. The memory 92 records road map dataindicating positions where the respective vehicle imaging cameras 10,10A, 10B, . . . are provided, and records updated road map data, forexample, in a case where the road map data is updated due toconstruction of a new road or road maintenance work.

The display unit 93 is configured by using a display device such as aliquid crystal display (LCD) or an organic electroluminescence (EL)display, and displays various pieces of data sent from the processorPRC2.

The processor PRC2 is configured by using, for example, a centralprocessing unit (CPU), a digital signal processor (DSP), or a fieldprogrammable gate array (FPGA), and functions as a control unit of theclient terminal 90, and performs a control process for integrallycontrolling operations of the respective units of the client terminal90, and data input and output processes, a data calculation process, anda data storage process with the respective units of the client terminal90. The processor PRC2 operates according to the program stored in thememory 92. The processor PRC2 displays a retrieval result (for example,refer to FIGS. 7, 8, 9, 10, 11, 12, 14, 15, 16A, and 16B) sent from thevideo accumulation/analysis server 50 or captured video data on thedisplay unit 93 by using the memory 92 during an operation thereof. Theprocessor PRC2 generates a retrieval request including a retrievalcondition corresponding to a UI operation signal, and sends theretrieval request to the video accumulation/analysis server 50 via thecommunication unit 91.

The processor PRC2 reads and executes the program stored in the memory92, and thus functionally realizes a reproduction unit (not illustrated)and a retrieval unit (not illustrated). The reproduction unit (notillustrated) outputs captured video data sent from the videoaccumulation/analysis server 50 to the display unit 93 such that thecaptured video data is reproduced, in response to an operation of theoperator OP1. The retrieval unit (not illustrated) generates a retrievalrequest including a retrieval condition that is input through anoperation of the operator OP1.

In the vehicle monitoring system 1A illustrated in FIG. 1B, aconfiguration difference from the vehicle monitoring system 1illustrated in FIG. 1A is a configuration of a videoaccumulation/analysis server 50A. In FIG. 1B, the videoaccumulation/analysis server 50A is not configured by a single serverunlike the video accumulation/analysis server 50 illustrated in FIG. 1A,and is configured by, for example, four servers. In other words, in FIG.1B, the video accumulation/analysis server 50A is configured by a singlenumber detection/collation server 51A having the numberdetection/collation core 51 illustrated in FIG. 1A, a single vehicletype/vehicle color detection server 52A having the vehicle type/vehiclecolor detection core 52 illustrated in FIG. 1A, a single facedetection/collation server 53A having the face detection/collation core53 illustrated in FIG. 1A, and a number-of-occupants measurement server54A having the number-of-occupants measurement core 54 illustrated inFIG. 1A.

Respective processes in the number detection/collation server 51A, thevehicle type/vehicle color detection server 52A, the facedetection/collation server 53A, and the number-of-occupants measurementserver 54A are the same as the corresponding processes in the numberdetection/collation core 51, the vehicle type/vehicle color detectioncore 52, the face detection/collation core 53, and thenumber-of-occupants measurement core 54, and thus detailed descriptionthereof will be omitted. In other words, the video accumulation/analysisserver 50 illustrated in FIG. 1A may not be configured by a singleserver, and may be configured by a plurality of servers in the samemanner as the video accumulation/analysis server 50A illustrated in FIG.1B. The same database as the database 57 of the videoaccumulation/analysis server 50 may be provided in each of the numberdetection/collation server 51A, the vehicle type/vehicle color detectionserver 52A, the face detection/collation server 53A, and thenumber-of-occupants measurement server 54A.

FIG. 2 is a block diagram illustrating an example of a hardwareconfiguration of each of the vehicle imaging cameras 10, 10A, 10B, . . .. As described above, the respective internal configurations of thevehicle imaging cameras 10, 10A, 10B, . . . may be the same as eachother except a configuration of the lens block 17 (refer to FIG. 2 ) ora configuration of the camera casing, and thus the vehicle imagingcamera 10 will be described as an example in FIG. 2 . The vehicleimaging camera 10 is configured to include a processor 11, a memory 12,a communication unit 13, an imaging element 14, a lighting driver 15, anighttime short-wavelength lighting 16, the lens block 17, a lens driver18, an in-lens filter switching module 19, an in-lens filter switchingdriver 20, a front filter switching module 21, a front filter switchingdriver 22, and an illuminance sensor S1.

The processor 11 is configured by using, for example, a CPU, a DSP, oran FPGA. The processor 11 functions as a controller controlling theoverall operation of the vehicle imaging camera 10, and performs acontrol process for controlling operations of the respective units ofthe vehicle imaging camera 10, and data input and output processes, adata calculation process, and a data storage process with the respectiveunits of the vehicle imaging camera 10. The processor 11 operatesaccording to the program stored in the memory 12. The processor 11temporarily stores data or information generated or acquired by theprocessor 11 into the memory 12 by using the memory 12 during anoperation thereof.

The memory 12 is configured by using, for example, a RAM and a ROM, andtemporarily preserves a program required to execute an operation of thevehicle imaging camera 10, and data or information generated during anoperation thereof. The RAM is, for example, a work memory used during anoperation of the vehicle imaging camera 10. The ROM stores and preservesin advance, for example, a program for controlling the vehicle imagingcamera 10.

The communication unit 13 can perform communication with the videoaccumulation/analysis server 50 via the network NW1 such as a wiredcommunication line or a wireless communication network (for example, awireless local area network (LAN)). The communication unit 13 delivers(transmits), for example, a captured video (also referred to as a“number plate video” in some cases) as a channel 1 under a first imagingcondition appropriate for imaging a number plate of a vehicle to thevideo accumulation/analysis server 50. The communication unit 13delivers (transmits) a captured video (also referred to as an“in-vehicle face video”) as a channel 2 under a second imaging conditionappropriate for imaging a face of an in-vehicle occupant of a vehicle tothe video accumulation/analysis server 50.

The communication unit 13 receives an external input signal transmittedfrom the client terminal 90 via the video accumulation/analysis server50, and outputs the external input signal to the processor 11. Theexternal input signal is, for example, a command for changing imageparameters for increasing or decreasing brightness of a captured videothrough an operation of the operator OP1 viewing the captured videodisplayed on the client terminal 90. Here, the image parameters are, forexample, an exposure time using an electronic shutter, a gain foramplifying an electrical signal for a captured image in the imagingelement 14, and the intensity of lighting from the nighttimeshort-wavelength lighting 16, and may not be limited thereto. Theprocessor 11 changes and sets a value of a corresponding image parameterin response to the external input signal. The set image parameter is setfor the imaging element 14 or the nighttime short-wavelength lighting 16according to a content thereof.

The imaging element 14 is, for example, an image sensor that can capturea high definition video of 2K, 4K, or 8K, and is configured by asolid-state imaging element such as a charged coupled device (CCD) or acomplementary metal oxide semiconductor (CMOS). The solid-state imagingelement generates an electrical signal for a captured imagecorresponding to an optical image on the basis of photoelectricconversion of the optical image formed on an imaging surface. Theimaging element 14 may be configured with an integrated circuit boardmounted with the solid-state imaging element, an amplifier amplifying anelectrical signal output from the solid-state imaging element, a gainadjustment unit that can adjust a gain (sensitivity) of the amplifier,an electronic shutter (simply referred to as a “shutter” in some cases)that can control an imaging time (so-called exposure time), and anexposure control circuit controlling an exposure time of the electronicshutter. An output from the imaging element 14 may be input to theprocessor 11 to be subjected to predetermined signal processing suchthat captured video data is generated, and a control circuit thatexecutes the predetermined signal processing in the imaging element 14such that the captured video data is generated may be provided.

The lighting driver 15 is configured by using, for example, a switchingcircuit switching turning-on and turning-off of each of a plurality ofnighttime short-wavelength lightings 16. The lighting driver 15 switcheseach of the plurality of nighttime short-wavelength lightings 16 toturning-on or turning-off in response to a control instruction from theprocessor 11. The lighting driver 15 may further include a variableamplification circuit that can adjust a light emission amount(intensity) of the nighttime short-wavelength lighting 16. In this case,the lighting driver 15 can perform light control by adjusting a lightemission amount (intensity) of the nighttime short-wavelength lighting16 in response to a control instruction from the processor 11.

The nighttime short-wavelength lighting 16 as an example of an IRlighting unit is configured by using, for example, a light emissiondiode (LED), and is disposed in a plurality (for example, ten totwenty). The nighttime short-wavelength lighting 16 applies IR light(near-infrared light) that is lighting light having an intensity basedon the control of the lighting driver 15 in order to enable a subject(for example, a vehicle) within an angle of view to be imaged when anoperation mode of the vehicle imaging camera 10 is a nighttime mode.

The lens block 17 includes a focus lens that forms an optical image of asubject on the imaging surface of the imaging element 14. The lens block17 may include not only the focus lens but also a zoom lens that canchange magnification from a telephoto to a wide angle.

The lens driver 18 is configured by using an electric circuit adjustinga position of at least one lens configuring the lens block 17. The lensdriver 18 adjusts a position of the focus lens of the lens block 17 inresponse to a control instruction from the processor 11. In a case wherethe lens block 17 has the zoom lens, the lens driver 18 may adjust themagnification of the zoom lens in response to a control instruction fromthe processor 11.

The in-lens filter switching module 19 is disposed on the rear side(that is, an opposite object side) of the lens block 17 and front side(that is, an object side) of the imaging element 14. The in-lens filterswitching module 19 is disposed to be able to switch an IR-cut filter(not illustrated) and a plain glass (not illustrated), and alternatelyswitches and disposes the IR-cut filter and the plain glass on anoptical axis of an optical system. The in-lens filter switching module19 disposes, for example, the IR-cut filter on the optical axis during adaytime mode. Consequently, during the daytime mode, red, green, andblue (RGB) light from which an IR band component is cut is received atthe imaging element 14, and thus a visible light image having favorablequality is obtained. On the other hand, the in-lens filter switchingmodule 19 disposes, for example, the plain glass on the optical axisduring a nighttime mode. Consequently, during the nighttime mode,incident light passing through the plain glass without the IR bandcomponent being cut by the IR-cut filter is received at the imagingelement 14, and an IR image having constant brightness (in other words,not too dark) is obtained on the basis of the received incident light.

The in-lens filter switching driver 20 is configured by using anelectric circuit driving the in-lens filter switching module 19. Thein-lens filter switching driver 20 drives the in-lens filter switchingmodule 19 to dispose either the IR-cut filter or the plain glass on theoptical axis in response to a control instruction from the processor 11.

The front filter switching module 21 alternately switches a band-passfilter and a polarization filter (for example, by slidably moving theband-pass filter and the polarization filter), and thus disposes theband-pass filter or the polarization filter on the optical axis. Thefront filter switching module 21 is disposed further toward a subjectside (that is, an object side) on the optical axis than the lens block17, and thus mechanical adjustment (for example, maintenance) of thefront filter switching module 21 is facilitated.

The front filter switching driver 22 is configured by using an electriccircuit driving a motor (not illustrated) for the front filter switchingmodule. The front filter switching driver 22 drives the motor for thefront filter switching module to move the front filter switching module21 such that the band-pass filter or the polarization filter is disposedon the optical axis, in response to a control instruction from theprocessor 11.

The illuminance sensor S1 as an example of a sensor detects theilluminance of ambient light of the vehicle imaging camera 10. Forexample, a photodiode or a phototransistor is used for the illuminancesensor S1. The illuminance sensor S1 is attached to a front surface of acasing of the vehicle imaging camera 10 such that the illuminance oflight in a direction in which a vehicle as a subject of the vehicleimaging camera 10 is present can be detected. Illuminance information(specifically, illuminance value data) detected by the illuminancesensor S1 is input to the processor 11. The processor 11 determineswhether an operation mode of the vehicle imaging camera 10 at thepresent time is the nighttime mode or the daytime mode on the basis ofthe illuminance information.

For example, in a case where it is determined that the illuminanceinformation is more than a default threshold value (that is, thesurrounding is bright), the processor 11 sets an operation mode of thevehicle imaging camera 10 to transition to the daytime mode. In a casewhere it is determined that the illuminance information is less than thedefault threshold value (that is, the surrounding is dark), theprocessor 11 sets an operation mode of the vehicle imaging camera 10 totransition to the nighttime mode. In a case where the illuminanceinformation at the present time is not more than or not less than thedefault threshold value, an operation mode at the present time ismaintained. Information (for example, a flag) indicating the daytimemode or the nighttime mode is temporarily preserved in, for example, thememory 12.

FIG. 3A is a flowchart illustrating an operation procedure examplerelated to imaging performed by the vehicle imaging cameras 10, 10A,10B, . . . . FIG. 3B is a flowchart illustrating an operation procedureexample related to data transmission performed by the vehicle imagingcameras 10, 10A, 10B, . . . . FIG. 3C is a flowchart illustrating anoperation procedure example related to lighting control performed by thevehicle imaging cameras 10, 10A, 10B, . . . . FIG. 3D is a tableillustrating various operation control examples in each of the daytimemode and the nighttime mode. In description of each of FIGS. 3A to 3D,the vehicle imaging camera 10 is also exemplified in the same manner.

The process illustrated in FIG. 3A is repeatedly executed, for example,from turning-on of a power supply of the vehicle imaging camera 10 toturning-off of the power supply thereof. In FIG. 3A, the processor 11calculates an image parameter under the first imaging condition or animage parameter under the second imaging condition on the basis of anexternal input signal from the communication unit 13 (SU), and sets thecalculation result (image parameter) for the memory 12. The vehicleimaging camera 10 switches and captures a number plate video and anin-vehicle face video in a time division manner (for example, everyframe). In this case, for example, the processor 11 captures a numberplate video (number plate image) as an odd-numbered frame (capturedimage), and captures an in-vehicle face video (in-vehicle face image) asan even-numbered frame (captured image).

After step St1, the processor 11 determines whether an exposure time ofthe electronic shutter for the imaging element 14 is “long” or “short”(St2).

The processor 11 determines that the exposure time of the electronicshutter corresponding to an in-vehicle face image is “long” when aneven-numbered frame is captured (Long in St2), and sets image parameters(for example, an exposure time of the electronic shutter and a gain)appropriate for capturing the in-vehicle face image for the imagingelement 14 (St3). The imaging element 14 transmits in-vehicle face imagedata to the processor 11 on the basis of the image parameters set instep St3 (St4). The processor 11 executes predetermined signalprocessing (for example, noise removal, white balancing, and imagecompression) on the in-vehicle face image data from the imaging element14 (St5), and temporarily accumulates the in-vehicle face image datasubjected to the signal processing into the memory 12 (St6). After stepSt6, the process in the vehicle imaging camera 10 returns to step St1.

On the other hand, the processor 11 determines that the exposure time ofthe electronic shutter corresponding to a number plate image is “short”when an odd-numbered frame is captured (Short in St2), and sets imageparameters (for example, an exposure time of the electronic shutter anda gain) appropriate for capturing the number plate image for the imagingelement 14 (St7). The imaging element 14 transmits number plate imagedata to the processor 11 on the basis of the image parameters set instep St7 (St8). The processor 11 executes predetermined signalprocessing (for example, noise removal, white balancing, and imagecompression) on the number plate image data from the imaging element 14(St9), and temporarily accumulates the number plate image data subjectedto the signal processing into the memory 12 (St6). Consequently, thevehicle imaging camera 10 can generate an in-vehicle face image capturedon the basis of image parameters appropriate for imaging a face of anoccupant in a vehicle and can generate a number plate image captured onthe basis of image parameters appropriate for imaging a number plate byswitching the image parameters for the in-vehicle face image and theimage parameters for the number plate image every frame.

The process illustrated in FIG. 3B is repeatedly executed, for example,from turning-on of the power supply of the vehicle imaging camera 10 toturning-off of the power supply thereof. In FIG. 3B, the processor 11reads and acquires the in-vehicle face video data or the number platevideo data that is temporarily accumulated in step St6 (St11).

The processor 11 determines whether an exposure time of the electronicshutter for the imaging element 14 corresponding to the video dataacquired in step St11 is “long” or “short” (St12).

In a case where it is determined that the exposure time is “long” (Longin St12), the processor 11 performs an encoding process appropriate forthe exposure time (St13). For example, in a case where an occupant'sface viewed through a windshield of a vehicle is acquired as a clearface image, the processor 11 performs the encoding process at a lowcompression ratio. The processor 11 delivers the in-vehicle face videodata subjected to the encoding process in step St13 as a stream 1 to thevideo accumulation/analysis server 50 via the communication unit 13(St14). After step St14, the process in the vehicle imaging camera 10returns to step St11.

On the other hand, in a case where it is determined that the exposuretime is “short” (Short in St12), the processor 11 performs an encodingprocess appropriate for the exposure time (St15). For example, in a casewhere a number plate image is acquired, the processor 11 may perform theencoding process at a high compression ratio. The processor 11 deliversthe number plate video data subjected to the encoding process in stepSt15 as a stream 2 to the video accumulation/analysis server 50 via thecommunication unit 13 (St16). After step St16, the process in thevehicle imaging camera 10 returns to step St11. Consequently, thevehicle imaging camera 10 can perform an encoding process at acompression ratio corresponding to in-vehicle face video data or numberplate video data according to an exposure time of the electronic shutterfor the imaging element 14, and can deliver the data to the videoaccumulation/analysis server 50.

A table TBL1 illustrated in FIG. 3D is registered in, for example, thememory 12 in advance. In the table TBL1, in the daytime mode, there arecommands for instruction information indicating that the front filterswitching module 21 disposes the polarization filter (not illustrated)on the optical axis, the in-lens filter switching module 19 disposes theIR-cut filter on the optical axis, and the nighttime short-wavelengthlighting 16 is turned off. On the other hand, in the nighttime mode,there are commands for instruction information indicating that the frontfilter switching module 21 disposes the band-pass filter (notillustrated) on the optical axis, the in-lens filter switching module 19disposes the plain glass on the optical axis, and the nighttimeshort-wavelength lighting 16 is turned on. Switching between the daytimemode and the nighttime mode illustrated in FIG. 3C is performed by theprocessor 11 on the basis of the command contents in the table TBL1.

The process illustrated in FIG. 3C is repeatedly executed, for example,from turning-on of a power supply of the vehicle imaging camera 10 toturning-off of the power supply thereof. In FIG. 3C, the processor 11acquires ambient illuminance information detected by the illuminancesensor S1 (St21). The processor 11 determines whether the present timeis the daytime (for example, the morning or the afternoon) or thenighttime (for example, the evening or the night) on the basis of theilluminance information acquired in step St21 (St22).

The processor 11 determines whether an operation mode of the vehicleimaging camera 10 is set to the daytime mode or the nighttime mode onthe basis of a determination result in step St22 (St23). For example,the memory 12 stores and preserves a threshold value regardingpredetermined illuminance. The processor 11 sets the operation mode tothe daytime mode in a case where the illuminance information is morethan the threshold value, and sets the operation mode to the nighttimemode in a case where the illuminance information is less than thethreshold value.

In a case of the daytime mode (daytime mode in St23), on the basis ofthe table TBL1 illustrated in FIG. 3D, the processor 11 generates acontrol instruction for executing a process during the daytime mode, andsends the control instruction to the front filter switching driver 22 soas to drive the front filter switching module 21 via the front filterswitching driver 22 (St24). In the daytime mode, the front filterswitching module 21 is moved such that the polarization filter (notillustrated) is located on the optical axis. The processor 11 sends thecontrol instruction for the daytime mode to the in-lens filter switchingdriver 20, and thus drives the in-lens filter switching module 19 viathe in-lens filter switching driver 20 (St25). In the daytime mode, thein-lens filter switching module 19 is moved such that the IR-cut filteris located on the optical axis in order to clarify an RGB image capturedby the imaging element 14.

The processor 11 similarly sends the control instruction for the daytimemode to the lighting driver 15, and thus turns off the plurality ofnighttime short-wavelength lightings 16 via the lighting driver 15(St26). After step St26, the process in the vehicle imaging camera 10returns to step St21.

On the other hand, in a case of the nighttime mode (nighttime mode inSt23), on the basis of the table TBL1 illustrated in FIG. 3D, theprocessor 11 generates a control instruction for executing a processduring the nighttime mode, and sends the control instruction to thefront filter switching driver 22 so as to drive the front filterswitching module 21 via the front filter switching driver 22 (St27). Inthe nighttime mode, the front filter switching module 21 is moved suchthat the band-pass filter (not illustrated) is located on the opticalaxis. The processor 11 sends the control instruction for the nighttimemode to the in-lens filter switching driver 20, and thus drives thein-lens filter switching module 19 via the in-lens filter switchingdriver 20 (St28). In the nighttime mode, the in-lens filter switchingmodule 19 is moved such that the plain glass is located on the opticalaxis in order not to cut IR light incident to the vehicle imaging camera10.

The processor 11 determines whether an exposure time of the electronicshutter for the imaging element 14 during imaging is “long” or “short”(St29).

The processor 11 determines that the exposure time of the electronicshutter corresponding to an in-vehicle face image is “long” when aneven-numbered frame is captured (Long in St29), and sends a controlinstruction for image parameters (for example, the intensity of IR lightfrom the nighttime short-wavelength lighting 16) appropriate forcapturing the in-vehicle face image during the nighttime mode to thelighting driver 15, so that the plurality of nighttime short-wavelengthlightings 16 are turned on in a pulse form via the lighting driver 15 onthe basis of the control instruction (St30A). After step St30A, theprocess in the vehicle imaging camera 10 returns to step St21.

On the other hand, the processor 11 determines that the exposure time ofthe electronic shutter corresponding to a number plate image is “short”when an odd-numbered frame is captured (Short in St29), and sends acontrol instruction for image parameters (for example, the intensity ofIR light from the nighttime short-wavelength lighting 16) appropriatefor capturing the number plate image during the nighttime mode to thelighting driver 15, so that the plurality of nighttime short-wavelengthlightings 16 are turned on in a pulse form via the lighting driver 15 onthe basis of the control instruction (St30B). After step St30B, theprocess in the vehicle imaging camera 10 returns to step St21.Consequently, since the vehicle imaging camera 10 adaptively switchesturning-on and turning-off of the front filter switching module 21, thein-lens filter switching module 19, and the nighttime short-wavelengthlighting 16 according to the daytime mode or the nighttime mode, highlyaccurate imaging can be achieved in both the daytime mode and thenighttime mode, and thus it is possible to generate an in-vehicle facevideo and a number plate video of a vehicle as a subject.

FIG. 4 is a block diagram illustrating a hardware configuration exampleof the video accumulation/analysis server 50 in detail. The videoaccumulation/analysis server 50 is configured to include a processorPRC1, a memory 55, a communication unit 56, and the database 57.

The processor PRC1 is configured by using, for example, a CPU, a DSP, oran FPGA. The processor PRC1 functions as a control unit of the videoaccumulation/analysis server 50, and performs a control process forintegrally controlling operations of the respective units of the videoaccumulation/analysis server 50, and data input and output processes, adata calculation process, and a data storage process with the respectiveunits of the video accumulation/analysis server 50. The processor PRC1operates according to the program stored in the memory 55. The processorPRC1 reads and executes the program stored in the memory 55, and thusfunctionally realizes the number detection/collation core 51, thevehicle type/vehicle color detection core 52, the facedetection/collation core 53, and the number-of-occupants measurementcore 54. Details of the number detection/collation core 51, the vehicletype/vehicle color detection core 52, the face detection/collation core53, and the number-of-occupants measurement core 54 have been describedwith reference to FIG. 1A, and thus description thereof will be omitted.

The memory 55 is configured by using, for example, a RAM and a ROM, andtemporarily stores a program or data required to execute an operation ofthe video accumulation/analysis server 50, and information or datagenerated during an operation thereof. The RAM is, for example, a workmemory used during an operation of the processor PRC1. The ROM stores inadvance, for example, a program and data for controlling the processorPRC1.

The communication unit 56 perform communication with each of the vehicleimaging cameras 10, 10A, 10B, . . . connected thereto via the networkNW1 such as a communication line of an intranet, and receives a capturedvideo (for example, a video indicating a status of a vehicle entering anangle of view) sent from each vehicle imaging camera. The communicationunit 56 performs communication with the client terminal 90 via a networksuch as an intranet provided in a police station, and receives aretrieval request sent from the client terminal 90 or returns a responseto the retrieval request.

The database 57 is configured by using, for example, a hard disk driveor a solid state drive. The database 57 records the captured video sentfrom each of the vehicle imaging cameras 10, 10A, 10B, . . . incorrelation with information regarding a camera ID of a camera havinggenerated the captured video and the imaging date and time. The database57 records road map data indicating positions where the respectivevehicle imaging cameras 10, 10A, 10B, . . . are provided, and recordsupdated road map data, for example, in a case where the road map data isupdated due to construction of a new road or road maintenance work.

The database 57 stores various processing results of a detectionprocess, a collation process, and a measurement process (analysisprocess) in the processor PRC1 in correlation with captured videos thatare targets of the processes.

The database 57 preserves a number collation list (an example of ablacklist) in which a number of a vehicle that is a police investigationtarget is registered in correlation with attribute information of theinvestigation target vehicle (for example, a stolen vehicle). The numbercollation list may be referred to, for example, when the numberdetection/collation core 51 performs a number collation process, and acontent thereof may be added, changed, or deleted to be updated asappropriate through an operation of the operator OP1.

The database 57 preserves a face collation list (an example of ablacklist) in which a face image of a person (for example, a suspect)who is a police investigation target is registered in correlation withattribute information of the investigation target person. The facecollation list may be referred to, for example, when the facedetection/collation core 53 performs a face image collation process, anda content thereof may be added, changed, or deleted to be updated asappropriate through an operation of the operator OP1.

Next, an operation procedure example regarding video analysis in thevehicle monitoring system 1 according to Embodiment 1 will be describedwith reference to FIGS. 5A, 5B, 5C, and 5D. FIG. 5A is a sequencediagram illustrating a first example of an operation procedure regardingvideo analysis in the vehicle monitoring system. FIG. 5B is a sequencediagram illustrating a second example of an operation procedureregarding video analysis in the vehicle monitoring system. FIG. 5C is asequence diagram illustrating a third example of an operation procedureregarding video analysis in the vehicle monitoring system. FIG. 5D is asequence diagram illustrating a fourth example of an operation procedureregarding video analysis in the vehicle monitoring system. Indescription of FIGS. 5A to 5D, the vehicle imaging camera 10 isexemplified in the same manner.

In the description of FIGS. 5A to 5D, for better understanding of thedescription, the configuration of the vehicle monitoring system 1illustrated in FIG. 1A will be described as an example. In thedescription of FIGS. 5B to 5D, the same process as a process in FIG. 5Awill be given the same step number, description thereof will besimplified or omitted, and different contents will be described.

In FIG. 5A, the vehicle imaging camera 10 does not have a function ofdetecting a number plate and a face reflected in a captured video. Thevehicle imaging camera 10 delivers (transmits) captured videos(specifically, an in-vehicle face video and a number plate video) to thevideo accumulation/analysis server 50 (St31). In a case where thecaptured videos are received from the vehicle imaging camera 10, thevideo accumulation/analysis server 50 (for example, the processor PRC1)sends the number plate video to the number detection/collation core 51(St32). The number detection/collation core 51 detects and recognizes anumber plate reflected in the number plate video, and collates thenumber plate recognition result with the number collation list preservedin the database 57 (St33). The number detection/collation core 51notifies the video accumulation/analysis server 50 of a result of thenumber plate collation process (St34).

The video accumulation/analysis server 50 sends the in-vehicle facevideo or the number plate video to the vehicle type/vehicle colordetection core 52 (St35). The vehicle type/vehicle color detection core52 detects a vehicle type and a vehicle color of a vehicle reflected inthe in-vehicle face video or the number plate video (St36), and notifiesthe video accumulation/analysis server 50 of detection results of thevehicle type and the vehicle color (St37).

In a case where there is a hit in the result of the number platecollation process (that is, matching with any number in the numbercollation list) or there are hits in the detection results of thevehicle type and the vehicle color (for example, matching with a vehicletype and a vehicle color of a runaway vehicle of a suspect to which theoperator OP1 has listened through an incoming call), the videoaccumulation/analysis server 50 sends the in-vehicle face video and aninstruction for executing face collation to the face detection/collationcore 53 (St38). The face detection/collation core 53 detects anoccupant's face reflected in the in-vehicle face video, and collates aface detection result with the face collation list preserved in thedatabase 57 (St39). The face detection/collation core 53 notifies thevideo accumulation/analysis server 50 of a result of the face collationprocess (St40).

In a case where there is a hit in the result of the number platecollation process (that is, matching with any number in the numbercollation list) or there are hits in the detection results of thevehicle type and the vehicle color (for example, matching with a vehicletype and a vehicle color of a runaway vehicle of a suspect to which theoperator OP1 has listened through an incoming call), the videoaccumulation/analysis server 50 sends the in-vehicle face video and aninstruction for measuring the number of occupants to thenumber-of-occupants measurement core 54 (St41). The number-of-occupantsmeasurement core 54 measures the number of occupants reflected in thein-vehicle face video (St42), and notifies the videoaccumulation/analysis server 50 of a measurement result of the number ofoccupants (St43).

The video accumulation/analysis server 50 accumulates (stores) thevarious processing results (specifically, the result of the number platecollation process, the detection results of the vehicle type and thevehicle color, the result of the face collation process, and themeasurement result of the number of occupants) of which notificationsare sent in steps St34, St37, St40, and St43 in the database 57 incorrelation with captured images (snapshots) configuring the capturedvideo received in step St31 (St44). The video accumulation/analysisserver 50 may send the various processing results and the snapshotsaccumulated in step St44, to the client terminal 90 (St45).Consequently, the operator OP1 operating the client terminal 90 canrecognize that various results of processes such as analysis on a videocaptured by the vehicle imaging camera 10 are accumulated in thedatabase 57 of the video accumulation/analysis server 50 and can thusalso visually check contents thereof via the display unit 93 of theclient terminal 90.

In FIG. 5B, the vehicle imaging camera 10 has a function of detecting anumber plate reflected in a captured video. The videoaccumulation/analysis server 50 (for example, the processor PRC1) sendsthe in-vehicle face video or the number plate video sent in step St31 tothe vehicle type/vehicle color detection core 52 and the facedetection/collation core 53 (St35A and St38A). Each of the vehicletype/vehicle color detection core 52 and the face detection/collationcore 53 discards the in-vehicle face video or the number plate videosent in step St31 until a notification (refer to step St53) of detectionof a number plate matching any number in the number collation list isreceived from the video accumulation/analysis server 50 (for example,the processor PRC1).

After step St31, the vehicle imaging camera 10 detects and recognizes anumber plate reflected in the number plate video (St51), and collates anumber plate recognition result with the number collation list preservedin the memory 12 (St51). The vehicle imaging camera 10 notifies thevideo accumulation/analysis server 50 of a result of the number platecollation process (St52). The video accumulation/analysis server 50sends a detection notification indicating that the number platereflected in the number plate video matches any number in the numbercollation list, to the vehicle type/vehicle color detection core 52 onthe basis of the result of the number plate collation process sent instep St52 (St53). The vehicle type/vehicle color detection core 52detects a vehicle type and a vehicle color of a vehicle reflected in thein-vehicle face video or the number plate video on the basis ofreception of the detection notification (St36). Consequently, theoperator OP1 operating the client terminal 90 can recognize that variousresults of processes such as analysis on a video captured by the vehicleimaging camera 10 are accumulated in the database 57 of the videoaccumulation/analysis server 50 and can thus also visually checkcontents thereof via the display unit 93 of the client terminal 90.Since the number plate collation process can be executed in the vehicleimaging camera 10 compared with the operation procedure in FIG. 5A,various processes on a video captured by the vehicle imaging camera 10can be distributed to the vehicle imaging camera 10 and the videoaccumulation/analysis server 50, and thus it is possible to reduce aprocessing load on the video accumulation/analysis server 50.

In FIG. 5C, the vehicle imaging camera 10 (for example, the processor11) has a function of detecting a number plate reflected in a capturedvideo, a vehicle color, and a face, and a function of measuring thenumber of occupants. After step St31, the vehicle imaging camera 10detects and recognizes a number plate reflected in the number platevideo, and collates a number plate recognition result with the numbercollation list preserved in the memory 12 (St51). The vehicle imagingcamera 10 detects a vehicle type and a vehicle color of a vehiclereflected in the in-vehicle face video or the number plate video(St36B).

In a case where there is a hit in the result of the number platecollation process (that is, matching with any number in the numbercollation list) or there are hits in the detection results of thevehicle type and the vehicle color (for example, matching with a vehicletype and a vehicle color of a runaway vehicle of a suspect to which theoperator OP1 has listened through an incoming call), the vehicle imagingcamera 10 detects an occupant's face reflected in the in-vehicle facevideo (St39B). The vehicle imaging camera 10 collates a face detectionresult with the face collation list preserved in the database 57(St39B).

In a case where there is a hit in the result of the number platecollation process (that is, matching with any number in the numbercollation list) or there are hits in the detection results of thevehicle type and the vehicle color (for example, matching with a vehicletype and a vehicle color of a runaway vehicle of a suspect to which theoperator OP1 has listened through an incoming call), the vehicle imagingcamera 10 measures the number of occupants reflected in the in-vehicleface video (St42B). The vehicle imaging camera 10 notifies the videoaccumulation/analysis server 50 of a result of the number platecollation process, detection results of the vehicle type and the vehiclecolor, a face dryness, and a measurement result of the number ofoccupants (St44B). Consequently, the operator OP1 operating the clientterminal 90 can recognize that various results of processes such asanalysis on a video captured by the vehicle imaging camera 10 areaccumulated in the database 57 of the video accumulation/analysis server50 and can thus also visually check contents thereof via the displayunit 93 of the client terminal 90. Since various processes such asnumber plate collation, detection of a vehicle type and a vehicle color,face collation, and measurement of the number of occupants can beexecuted in the vehicle imaging camera 10 compared with the operationprocedures in FIGS. 5A and 5B, the high performance vehicle imagingcamera 10 is used, and thus it is possible to remarkably reduce aprocessing load on the video accumulation/analysis server 50.

In FIG. 5D, the vehicle imaging camera 10 (for example, the processor11) has a function of detecting a number plate and a face reflected in acaptured video. The video accumulation/analysis server 50 (for example,the processor PRC1) sends the in-vehicle face video or the number platevideo sent in step St31 to the vehicle type/vehicle color detection core52 (St35A), and sends the in-vehicle face video sent in step St31 to thenumber-of-occupants measurement core 54 (St41C). The vehicle imagingcamera 10 detects a number plate reflected in the number plate video(St61), and generates a number plate image (hereinafter, referred to asa “number image” in some cases) obtained by cutting out the portion ofthe number plate reflected in the number plate video (St61). The vehicleimaging camera 10 generates a detection information notificationindicating that the number plate reflected in the number plate video hasbeen detected, and sends the detection information notification and thenumber plate image generated in step St61 to the videoaccumulation/analysis server 50 in correlation with each other (St62).

The video accumulation/analysis server 50 sends the detectioninformation notification and the number plate image to the numberdetection/collation core 51 in correlation with each other (St63). Thenumber detection/collation core 51 collates the number plate image (thatis, the image of the cutout number plate) with the number collation listpreserved in the database 57 (St33C).

The video accumulation/analysis server 50 sends the detectioninformation notification and the number plate image to the vehicletype/vehicle color detection core 52 and the number-of-occupantsmeasurement core 54 in correlation with each other (St64). The vehicletype/vehicle color detection core 52 detects a vehicle type and avehicle color of a vehicle reflected in the number plate video or thein-vehicle face video sent in step St35A (St36). The number-of-occupantsmeasurement core 54 measures the number of occupants reflected in thein-vehicle face video sent in step St41C (St42).

The vehicle imaging camera 10 detects an occupant's face reflected inthe in-vehicle face video (St65), and generates a face image obtained bycutting out the portion of the face reflected in the in-vehicle facevideo (St65). The vehicle imaging camera 10 generates a detectioninformation notification indicating that the occupant's face reflectedin the in-vehicle face video has been detected, and sends the detectioninformation notification and the face image generated in step St65 tothe video accumulation/analysis server 50 in correlation with each other(St66).

The video accumulation/analysis server 50 sends the detectioninformation notification and the face image to the facedetection/collation core 53 in correlation with each other (St67). Theface detection/collation core 53 collates the face image (that is, theimage of the cutout occupant's face) with the face collation listpreserved in the database 57 (St39). The face detection/collation core53 notifies the video accumulation/analysis server 50 of a result of theface collation process (St40C). The number detection/collation core 51notifies the video accumulation/analysis server 50 of a result of thenumber plate collation process (St34C). The vehicle type/vehicle colordetection core 52 notifies the video accumulation/analysis server 50 ofdetection results of the vehicle type and the vehicle color (St37C). Thenumber-of-occupants measurement core 54 notifies the videoaccumulation/analysis server 50 of a measurement result of the number ofoccupants (St43C).

The video accumulation/analysis server 50 accumulates (stores) thevarious processing results (specifically, the result of the number platecollation process, the detection results of the vehicle type and thevehicle color, the result of the face collation process, and themeasurement result of the number of occupants) of which notificationsare sent in steps St34C, St37C, St40C, and St43C in the database 57 incorrelation with captured images (snapshots) configuring the capturedvideo received in step St31 (St44). Consequently, the operator OP1operating the client terminal 90 can recognize that various results ofprocesses such as analysis on a video captured by the vehicle imagingcamera 10 are accumulated in the database 57 of the videoaccumulation/analysis server 50 and can thus also visually checkcontents thereof via the display unit 93 of the client terminal 90. Anumber plate image and a face image that are targets of collationprocesses with the number collation list and the face collation list aregenerated by cutting out only a corresponding number plate and face fromcaptured videos, and thus it is possible to reduce loads of a numberplate collation process and a face collation process.

Next, with reference to FIGS. 6A and 6B, a description will be made ofan example of a process in which the vehicle imaging camera 10 registersa driver and a passenger. FIG. 6A is a flowchart illustrating a firstexample of an operation procedure of a process in which the vehicleimaging camera 10 registers a driver and a passenger. FIG. 6B is aflowchart illustrating a second example of an operation procedure of aprocess in which the vehicle imaging camera 10 registers a driver and apassenger. The processes in FIGS. 6A and 6B will be described to beexecuted by, for example, the vehicle imaging camera 10 in the processin step St65 in FIG. 5D, but may be executed by the facedetection/collation core 53 in the process in step St39 in FIGS. 5A and5B, and may be executed by the vehicle imaging camera 10 in the processin step St39B in FIG. 5C. Similarly, in description of FIGS. 6A and 6B,the vehicle imaging camera 10 is exemplified, and an example is supposedin which a steering wheel is attached on the right side of a vehicle asa subject in an advancing direction.

In FIG. 6A, the vehicle imaging camera 10 (for example, the processor11) detects an occupant's face from a captured video (in-vehicle facevideo), and generates a face image obtained by cutting out the portionof the face reflected in the in-vehicle face video (St65-1). The vehicleimaging camera 10 determines whether or not the number of detected facesis one (in other words, whether or not a plurality of persons areriding) on the basis of the processing result in step St65-1 (St65-2).In a case where it is determined that the number of detected faces isone (YES in St65-2), the vehicle imaging camera 10 registers the faceimage to have an attribute as a driver (St65-3). After step St65-3, theprocess illustrated in FIG. 6A is finished.

On the other hand, in a case where it is determined that the number ofdetected faces is not one (NO in St65-2), the vehicle imaging camera 10detects positions of faces reflected in the in-vehicle face video(St65-4). The vehicle imaging camera 10 determines whether or not thedetected face is a face on the left side when viewed from the vehicleimaging camera 10 (St65-5). In a case where it is determined that thedetected face is not a face on the left side when viewed from thevehicle imaging camera 10 (NO in St65-5), the vehicle imaging camera 10registers the face image to have an attribute as a passenger (forexample, a person on a passenger seat) (St65-6). After step St65-6, theprocess illustrated in FIG. 6A is finished.

On the other hand, in a case where it is determined that the detectedface is a face on the left side when viewed from the vehicle imagingcamera 10 (YES in St65-5), the vehicle imaging camera 10 registers theface image to have an attribute as a driver (St65-7). After step St65-7,the process illustrated in FIG. 6A is finished.

In FIG. 6B, the vehicle imaging camera 10 (for example, the processor11) detects an occupant's face from a captured video (in-vehicle facevideo), and generates a face image obtained by cutting out the portionof the face reflected in the in-vehicle face video (St65-1). The vehicleimaging camera 10 detects a steering wheel in the vehicle from thein-vehicle face video (St65-8), and determines whether or not thesteering wheel detected in step St65-8 is present near the face detectedin step St65-1 (St65-9).

In a case where it is determined that the steering wheel is not presentnear the face (NO in St65-9), the vehicle imaging camera 10 registersthe face image to have an attribute as a passenger (for example, aperson on a passenger seat) (St65-6). After step St65-6, the processillustrated in FIG. 6B is finished.

On the other hand, in a case where it is determined that the steeringwheel is present near the face (YES in St65-9), the vehicle imagingcamera 10 registers the face image to have an attribute as a driver(St65-7). After step St65-7, the process illustrated in FIG. 6B isfinished. Consequently, through the process in FIG. 6A or 6B, it can beappropriately determined whether a face reflected in an in-vehicle facevideo is an occupant's face or a passenger's face in the analysisprocess (for example, FIGS. 5A to 5D) performed by the vehicle imagingcamera 10 or the video accumulation/analysis server 50. Therefore, aneffect is expected in which intensive retrieval can be performed in thevideo accumulation/analysis server 50, and thus the operator OP1operating the client terminal 90 can easily narrow a suspect. Forexample, in a case where retrieval using retrieval conditions includingattributes of a “driver” and a “passenger” cannot be performed, there isa high probability that both an in-vehicle face video in which a suspectis a “driver” and an in-vehicle face video in which the identicalsuspect is a “passenger” are retrieval targets such that a large numberof retrieval results are extracted, and thus it takes time to narrow thesuspect. However, since retrieval including attributes of a “driver” anda “passenger” as retrieval conditions can be performed, only anin-vehicle face video in a case where a suspected person is a “driver”can be narrowed to a retrieval target, and thus the accuracy of aretrieval result is improved.

Next, with reference to FIGS. 7 to 12, and 14 to 16B, description willbe made of various screen examples displayed on the display unit 93 ofthe client terminal 90 in police investigation using the vehiclemonitoring system 1. In description of FIGS. 7 to 12, and 14 to 16B, thesame constituent element as a constituent element illustrated in thedrawings will be given the same reference numeral, and descriptionthereof will be simplified or omitted.

In police investigation, the client terminal 90 is starting andexecuting a vehicle monitoring application that is installed in advance,through an operator's operation. The vehicle monitoring application isstore in, for example, the ROM of the memory 92 of the client terminal90, and is executed by the processor PRC2 when started through theoperator's operation. Data or information generated by the processorPRC2 during starting of the vehicle monitoring application istemporarily preserved in the RAM of the memory 92.

FIG. 7 is a diagram illustrating a monitoring screen example of acaptured video in the daytime mode. As illustrated in FIG. 7 , amonitoring screen WD1 includes at least a display region of an entryfield COND1 to which a retrieval condition is entered, a display regionof a list display field EVE1 in which retrieval results of events (forexample, vehicles) satisfying a retrieval condition are displayed in alist form, a display region of a details display field DETL1 in whichdetails of an event selected with a cursor CUR1 are displayed, and aretrieval icon SCH1.

The entry field COND1 includes, for example, a start time entry fieldFR1, an end time entry field TO1, a camera selection field CM1, and atext entry field TXT1 for a retrieval target number plate.

The date and time of starting of the presence of a retrieval targetrunaway vehicle are entered to the start time entry field FR1 through anoperation of the operator OP1 in order to cause the videoaccumulation/analysis server 50 to retrieve the runaway vehicle of asuspect. For example, the date and time of the occurrence of a case orthe like or the date and time immediately before the date and time areentered to the start time entry field FR1. FIGS. 7 to 9 illustrate anexample in which “2:00 AM on Apr. 23, 2018” is entered to the start timeentry field FR1. In a case where the date and time are entered throughan operation of the operator OP1, the vehicle monitoring applicationsets the date and time entered to the start time entry field FR1 as aretrieval condition (for example, start date and time).

The date and time of ending of the presence of a retrieval targetrunaway vehicle are entered to the end time entry field TO1 through anoperation of the operator OP1 in order to cause the videoaccumulation/analysis server 50 to retrieve the runaway vehicle of asuspect. For example, end date and time of a retrieval period for arunaway vehicle are entered to the end time entry field TO1. FIGS. 7 to9 illustrate an example in which “12:00 AM on Apr. 24, 2018” is enteredto the end time entry field TO1. In a case where the date and time areentered through an operation of the operator OP1, the vehicle monitoringapplication sets the date and time entered to the end time entry fieldTO1 as a retrieval condition (for example, end date and time).

In the camera selection field CM1, a vehicle imaging camera at alocation where the retrieval target runaway vehicle is predicted to bereflected is designated through an operation of the operator OP1 inorder to cause the video accumulation/analysis server 50 to retrieve therunaway vehicle of the suspect.

Text data regarding a number plate of the retrieval target runawayvehicle is entered to the text entry field TXT1 through an operation ofthe operator OP1 in order to cause the video accumulation/analysisserver 50 to retrieve the number plate of the runaway vehicle of thesuspect. As illustrated in FIG. 7 , in the text entry field TXT1, thecharacter “%” may be used for wild card search, and, for example, a noteis displayed in which “A %” is entered in retrieval of a number platestarting with the character “A”.

The retrieval icon SCH1 is displayed by the vehicle monitoringapplication to be able to be pressed at the time at which variousretrieval conditions in the entry field COND1 entered through theoperations of the operator OP1 are all appropriately entered. In a casewhere the retrieval icon SCH1 is pressed through an operation of theoperator OP1, the vehicle monitoring application detects the pressing,generates a vehicle number plate retrieval request including the enteredvarious retrieval conditions, and sends the request to the videoaccumulation/analysis server 50 via the communication unit 91. Thevehicle monitoring application receives number plate retrieval resultsfrom the video accumulation/analysis server 50, and displays theretrieval results in the list display field EVE1.

A plurality of retrieval results each including at least, for example,the date and time on which the number plate was detected by the numberdetection/collation core 51, the name of a camera used to detect thenumber plate, and a reading result (text data) of the number plate(license plate) are arranged in a list form in the list display fieldEVE1.

In a case where any one is selected with the cursor CUR1 from among theplurality of retrieval results arranged in a list form through anoperation of the operator OP1, the vehicle monitoring applicationdisplays details information (including a number plate image) regardingthe selected retrieval result in the details display field DETL1.Specifically, the vehicle monitoring application displays the date andtime of detection of the number plate selected with the cursor CUR1, thename of the camera used to detect the number plate, a number plate imageLPcap1 in which the whole vehicle corresponding to the detected numberplate is reflected, number plate attribute information LPIF1 including anumber plate cutout image LPEX1, an event moving image viewer icon VEV1(refer to FIGS. 8 and 9 ), and a period designation icon TAJ1 (refer toFIGS. 8 and 9 ) in correlation with each other.

The number plate attribute information LPIF1 includes the number platereading result (text data) and the number plate cutout image LPEX1, acountry name or a state name, a color of the number plate, and a score(similarity) during retrieval (specifically, number plate collation).Consequently, the operator OP1 just selects a suspected number platefrom the list display field EVE1, and can thus visually recognizedetailed information regarding the number plate along with the numberplate image LPcap1 in which a vehicle attached with the number plate isreflected such that the efficiency of investigation can be improved.

FIG. 8 is a diagram illustrating a monitoring screen example ofdisplaying a moving image reproduction screen MVFC1 in the daytime modein which an in-vehicle occupant is clearly reflected and a moving imagereproduction screen MVLP1 in the daytime mode in which a number plate isclearly reflected. FIG. 9 is a diagram illustrating a monitoring screenexample of displaying a moving image reproduction screen MVFC2 in thenighttime mode in which an in-vehicle occupant is clearly reflected anda moving image reproduction screen MVLP2 in the nighttime mode in whicha number plate is clearly reflected. The monitoring screen WD1illustrated in FIGS. 8 and 9 are also displayed on the display unit 93of the client terminal 90 in the same manner as the monitoring screenWD1 illustrated in FIG. 7 .

The vehicle monitoring application displays the moving imagereproduction screens MVFC1 and MVLP1 for an in-vehicle face video and anumber plate video captured at the substantially same time (for example,during the daytime mode) by the same vehicle imaging camera 10corresponding to the number plate image LPcap1 through the operator'soperation of pressing the event moving image viewer icon VEV1 in a statein which any one retrieval result (that is, the number plate) isselected with the cursor CUR1 (refer to FIG. 8 ). The vehicle monitoringapplication can set moving image reproduction periods for the in-vehicleface video and the number plate video with the period designation iconTAJ1, and FIG. 8 illustrates that “5 seconds” is displayed as a movingimage reproduction period. The vehicle monitoring application displaysoperation control icons BTN1 and BTN2 such as a play, a pause, a fastforward, and a fast rewind of the moving image reproduction screensMVFC1 and MVLP1.

In FIG. 8 , the moving image reproduction screens MVFC1 and MVLP1 forvideos (specifically, an in-vehicle face video and a number plate video)captured by the vehicle imaging camera 10 in the daytime mode aredisplayed. Consequently, the in-vehicle face video in the moving imagereproduction screen MVFC1 is reproduced such that a face of anin-vehicle occupant (for example, a driver) is clearly reflected eventhough the surrounding is bright in a daytime period, and thus it ispossible to efficiently assist the operator OP1 in recognizing the faceof the occupant in an early stage. On the other hand, the number platevideo in the moving image reproduction screen MVLP1 is reproduced suchthat a number plate of a vehicle is clearly reflected even though thesurrounding is bright in a daytime period, and thus it is possible toefficiently assist the operator OP1 in recognizing the number plate inan early stage.

The vehicle monitoring application displays the moving imagereproduction screens MVFC2 and MVLP2 for an in-vehicle face video and anumber plate video captured at the substantially same time (for example,during the nighttime mode) by the same vehicle imaging camera 10corresponding to the number plate image LPcap2 through the operator'soperation of pressing the event moving image viewer icon VEV1 in a statein which any one retrieval result (that is, the number plate) isselected with the cursor CUR1 (refer to FIG. 9 ). The vehicle monitoringapplication can set moving image reproduction periods for the in-vehicleface video and the number plate video with the period designation iconTAJ1, and FIG. 9 illustrates that “5 seconds” is displayed as a movingimage reproduction period. The vehicle monitoring application displaysthe operation control icons BTN1 and BTN2 such as a play, a pause, afast forward, and a fast rewind of the moving image reproduction screensMVFC2 and MVLP2.

In FIG. 9 , the moving image reproduction screens MVFC2 and MVLP2 forvideos (specifically, an in-vehicle face video and a number plate video)captured by the vehicle imaging camera 10 in the nighttime mode aredisplayed. Consequently, the in-vehicle face video in the moving imagereproduction screen MVFC2 is reproduced such that a face of anin-vehicle occupant (for example, a driver) is clearly reflected eventhough the surrounding is dark in a nighttime period, and thus it ispossible to efficiently assist the operator OP1 in recognizing the faceof the occupant in an early stage. The number plate video in the movingimage reproduction screen MVLP2 is reproduced such that a number plateof a vehicle is clearly reflected even though the surrounding is dark ina nighttime period, and thus it is possible to efficiently assist theoperator OP1 in recognizing the number plate in an early stage.

Retrieval of Face Image Based on Number Plate

FIG. 10 is a diagram illustrating an example of a monitoring screendisplaying a retrieval result of a face image of an in-vehicle occupantusing a number plate that is input as a retrieval condition. FIG. 11A isa diagram illustrating a first example of a monitoring screen displayedsubsequently to the monitoring screen in FIG. 10 . FIG. 11B is a diagramillustrating a second example of a monitoring screen displayedsubsequently to the monitoring screen in FIG. 10 . FIG. 12 is a diagramillustrating a third example of a monitoring screen displayedsubsequently to the monitoring screen in FIG. 10 . The monitoringscreens respectively illustrated FIGS. 10 to 12 are examples ofretrieval result screens displayed on the client terminal 90 in asequence diagram of FIG. 13 .

A captured video illustrated in FIG. 10 is a number plate image LPcap3captured under a first imaging condition appropriate for imaging anumber plate NBPL1, and is used to retrieve a face image based on anumber plate. For example, two occupants (specifically, a driver and apassenger) are riding on the vehicle VCL1 reflected in the number plateimage LPcap3.

In a case where the number plate image LPcap3 displayed on the displayunit 93 of the client terminal 90 is viewed, the operator OP1 may wantto retrieve face images FCE2 and FCE3 of the occupants riding on thevehicle VCL1 having the number plate NBPL1. An operation procedure inthe vehicle monitoring system 1 in a case of retrieving the face imagesFCE2 and FCE3 of the occupants in the vehicle VCL1 by using the numberplate NBPL1 will be described in detail with reference to FIG. 13 whichwill be described later. In a case where such retrieval is performed, aretrieval result screen WD2 illustrated in FIG. 10 is displayed on thedisplay unit 93 of the client terminal 90.

The retrieval result screen WD2 illustrated in FIG. 10 includes at leasta start time entry field for start date and time and an end time entryfield for end date and time, the start date-and-time information and theend date and time being used for retrieval, a face image display fieldFCC displaying face images as retrieval results, a number-of-occupantsmeasurement result display field HCC displaying the number of occupantsas a retrieval result, a vehicle details information display field VCC,and a plurality of candidate persons' face images list display fieldLST2. The start time entry field and the end time entry field arerespectively the same as the start time entry field FR1 and the end timeentry field TO1 illustrated in FIG. 7 , and thus description thereofwill be omitted here.

A face image of a person having the highest score (that is, similarity)as a result of retrieving a face image of an occupant of a vehicle byusing a number plate is displayed in the face image display field FCC.In the example illustrated in FIG. 10 , the face images FCE2 and FCE3 ofoccupants of the vehicle VCL1 in the number plate image LPcap3 aredisplayed.

A measurement result NUB1 of the number of occupants of the vehicle VCL1in the number plate image LPcap3 is displayed in the number-of-occupantsmeasurement result display field HCC. In the example illustrated in FIG.10 , “two persons” is displayed to correspond to the number of faceimages displayed in the face image display field FCC.

A snapshot (for example, a captured image configuring the number plateimage LPcap3) of the vehicle VCL1 in the number plate image LPcap3 andvarious pieces of vehicle information VCLIF1 (including the number plateNBPL1) regarding the vehicle VCL1 are displayed in the vehicle detailsinformation display field VCC. The vehicle information VCLIF1 includes,for example, data (hereinafter, referred to as “vehicle registrationinformation”) such as an owner of a vehicle registered in the LandTransport Bureau or the like, information indicating whether or not thevehicle is a stolen vehicle, information indicating the presence orabsence of the past accident, and vehicle inspection information, and isnot limited thereto.

A list of face images of a plurality of candidate persons which may bethe occupant of the vehicle VCL1 in the number plate image LPcap3 isdisplayed in the candidate persons' face images list display field LST2.In a case where a matching face image is identified from the facecollation list (refer to the database 57) in the process of collatingface images of the occupant of the vehicle VCL1 in the number plateimage LPcap3 with each other (refer to FIG. 13 ), the vehicle monitoringapplication may display a popup screen CPR1. The popup screen CPR1displays a face image RGFC1 registered in the face collation list, theface image FCE2 reflected in the number plate image LPcap3, the imagingdate and time and a camera ID (in other words, information regarding aninstallation position of the corresponding vehicle imaging camera 10) ofthe vehicle imaging camera 10 having captured the number plate imageLPcap3, in order to present, to the operator OP1, that the face imageFCE2 is registered in the face collation list (for example, a blacklistin which face images of ex-convicts or the like are registered).

A retrieval result screen WD3 illustrated in FIG. 11A is displayed bythe vehicle monitoring application due to an operation of the operatorOP1 on the retrieval result screen WD2. The retrieval result screen WD3includes at least a start time entry field for start date and time andan end time entry field for end date and time, the start date-and-timeinformation and the end date and time being used for retrieval, the faceimage display field FCC displaying face images as retrieval results, thenumber-of-occupants measurement result display field HCC displaying thenumber of occupants as a retrieval result, the vehicle detailsinformation display field VCC, and time line display fields TML1 andTML2.

The time line display field TML1 displays, in a time series, the dateand time on which a person corresponding to a face image (for example,the face image FCE3) displayed in the face image display field FCC wasdetected by the vehicle imaging camera. In the example illustrated inFIG. 11A, the person corresponding to the face image FCE3 is detectedbetween “12:20:30” and “12:20:34” by a vehicle imaging camera having thename of “LPR Camera”.

The time line display field TML2 displays, in a time series, the dateand time on which a person corresponding to a face image (for example,the face image FCE2) displayed in the face image display field FCC wasdetected by the vehicle imaging camera. In the example illustrated inFIG. 11A, the person corresponding to the face image FCE2 is detectedbetween “12:20:30” and “12:20:34” by a vehicle imaging camera having thename of “LPR Camera2”.

A retrieval result screen WD4 illustrated in FIG. 11B is displayed bythe vehicle monitoring application due to an operation of the operatorOP1 on the retrieval result screen WD2. The retrieval result screen WD4includes at least a start time entry field for start date and time andan end time entry field for end date and time, the start date-and-timeinformation and the end date and time being used for retrieval, the faceimage display field FCC displaying face images as retrieval results, thenumber-of-occupants measurement result display field HCC displaying thenumber of occupants as a retrieval result, the vehicle detailsinformation display field VCC, and a time line display field TML3. Theretrieval result screen WD4 exemplifies a case where only the face imageFCE3 is identified as a face image as a retrieval result.

The time line display field TML3 displays, in a time series, the dateand time on which a person corresponding to a face image (for example,the face image FCE3) displayed in the face image display field FCC wasdetected by each of a plurality of vehicle imaging cameras, a face image(limited to a case where the face image is registered) registered in theface collation list, and a score (similarity) regarding the detection incorrelation with each other. In the example illustrated in FIG. 11B, theperson corresponding to the face image FCE3 is detected with the score“90” by a vehicle imaging camera having the name of “LPR Camera” at“15:42:31 on Apr. 23, 2018”. The person corresponding to the face imageFCE3 is detected with the score “70” by a vehicle imaging camera havingthe name of “LPR Camera2” at “15:40:20 on Apr. 23, 2018”. The personcorresponding to the face image FCE3 is detected with the score “65” bythe vehicle imaging camera having the name of “LPR Camera” at “15:30:20on Apr. 23, 2018”. The person corresponding to the face image FCE3 isdetected with the score “50” by the vehicle imaging camera having thename of “LPR Camera2” at “15:37:57 on Apr. 23, 2018”. The personcorresponding to the face image FCE3 is detected with the score “30” bythe vehicle imaging camera having the name of “LPR Camera” at “15:36:10on Apr. 23, 2018”. The person corresponding to the face image FCE3 isdetected with the score “25” by the vehicle imaging camera having thename of “LPR Camera” at “15:35:20 on Apr. 23, 2018”.

A retrieval result screen WD5 illustrated in FIG. 12 is displayed by thevehicle monitoring application due to an operation of the operator OP1on the retrieval result screen WD2. The retrieval result screen WD5displays escape routes DRC1 and DRC2 of the vehicle VCL1 in the numberplate image LPcap3 to be superimposed on road map data MP1 along with avehicle icon ICO1 indicating the vehicle VCL1. In a case where it isdetected that any one of intersections (refer to circular marks in FIG.12 ) where the vehicle imaging cameras are provided is designatedthrough an operation of the operator OP1, the vehicle monitoringapplication may display a moving image reproduction screen for thenumber plate image LPcap3 immediately before the vehicle passes throughthe designated intersection to be superimposed on the retrieval resultscreen WD5.

In a case where it is detected that the escape route DRC1 is designatedthrough an operation of the operator OP1, the vehicle monitoringapplication may display the number plate NBPL1 and the face image FCE2of the driver reflected in the number plate image LPcap3 captured by avehicle imaging camera provided at an intersection corresponding to astart point or an end point of the escape route DRC1 to be superimposedon each other. Similarly, in a case where it is detected that the escaperoute DRC2 is designated through an operation of the operator OP1, thevehicle monitoring application may display the number plate NBPL1 andthe face image FCE2 of the driver reflected in the number plate imageLPcap3 captured by a vehicle imaging camera provided at an intersectioncorresponding to a start point or an end point of the escape route DRC2to be superimposed on each other. In this case, the vehicle monitoringapplication may display a balloon indicating the name of an imaginglocation corresponding to an installation location of the vehicleimaging camera on the road map data MP1.

Next, with reference to FIG. 13 , a description will be made of a seriesof operation procedures of retrieving a face image of an in-vehicleoccupant by using a number plate in the vehicle monitoring system 1according to Embodiment 1. FIG. 13 is a sequence diagram illustrating anexample of a series of operation procedures of retrieving a face imageof an in-vehicle occupant by using a number plate that is input as aretrieval condition. In the description of FIG. 13 , similarly, aconfiguration of a vehicle imaging camera exemplifies the configurationof the vehicle monitoring system 1 illustrated in FIG. 1A, and aconfiguration of a vehicle imaging camera exemplifies the configurationof the vehicle imaging camera 10. In the description of FIG. 13 , it isassumed that, for example, as illustrated in FIG. 5A, whenever a numberplate video or an in-vehicle face video is received, the videoaccumulation/analysis server 50 acquires a result of a number platecollation process, detection results of a vehicle type and a vehiclecolor, a result of a face collation process, and a measurement result ofthe number of occupants, based on the number plate video or thein-vehicle face video (St34, St37, St40, and St43), and accumulates theresults (St44).

In FIG. 13 , in a case where a number plate (for example, the numberplate NBPL1) that is input through an operation of the operator OP1 isinput as a retrieval condition (St71), the video accumulation/analysisserver 50 receives a retrieval request (including an image of the numberplate NBPL1) sent from the client terminal 90. In response to thereception of the retrieval request, the video accumulation/analysisserver 50 (for example, the processor PRC1) sends a number collationrequest for instructing a collation process for the number plate NBPL1to be executed to the number detection/collation core 51 (St72). Thenumber detection/collation core 51 collates the number plate NBPL1 withthe number collation list preserved in the database 57 (St73). Thenumber detection/collation core 51 notifies the videoaccumulation/analysis server 50 of results (for example, the numberplate NBPL1, a camera ID, an imaging time point, and vehicleregistration information (refer to the above description)) of the numberplate collation process (St74).

The video accumulation/analysis server 50 retrieves (extracts)corresponding accumulated information (for example, the face image ofthe detected driver or passenger, the vehicle type and the vehiclecolor, and the result of the number of occupants) from the accumulatedresults of the various processes accumulated in step St44 on the basisof the camera ID and the imaging time point included in the processingresults obtained in step St74 (St75).

The video accumulation/analysis server 50 sends the face image of thedriver or the passenger extracted in step St75 and a face collationrequest to the face detection/collation core 53 (St76). The facedetection/collation core 53 collates the face image of the driver or thepassenger with the face collation list preserved in the database 57(St77). The face detection/collation core 53 notifies the videoaccumulation/analysis server 50 of results (for example, a score, acamera ID, and a list of imaging time points) of the face collationprocess (St78).

The video accumulation/analysis server 50 lists scores and accumulatedresults associated with the camera IDs and the imaging time points ofwhich notifications are sent in steps St74 and St78 (St79). The videoaccumulation/analysis server 50 sends a result list including theaccumulated results (for example, number information, vehicleregistration information, a face image, a face collation result,information regarding a vehicle type and a vehicle color, andinformation regarding the number of occupants) to the client terminal 90as a retrieval result along with the scores (St80). The client terminal90 displays the retrieval result screen WD2 (for example, refer to FIG.10 ) by using the retrieval result sent in step St80 (St81).

Retrieval of Number Plate Based on Face Image

FIG. 14 is a diagram illustrating an example of a retrieval resultscreen displaying a retrieval result of a number plate that is retrievedby using a face image of an in-vehicle occupant that is input as aretrieval condition. FIG. 15 is a diagram illustrating a first exampleof a retrieval result screen displayed subsequently to the retrievalresult screen in FIG. 14 . FIG. 16A is a diagram illustrating a secondexample of a retrieval result screen displayed subsequently to theretrieval result screen in FIG. 14 . FIG. 16B is a diagram illustratinga third example of a retrieval result screen displayed subsequently tothe retrieval result screen in FIG. 14 . Monitoring screens respectivelyillustrated in FIGS. 14 to 16B are examples of retrieval result screensdisplayed on the client terminal 90 in a sequence diagram of FIG. 17 .

A captured video illustrated in FIG. 14 may be an in-vehicle face videocaptured under a second imaging condition appropriate for capturing aface image, and the in-vehicle face video is used to retrieve a numberplate based on a face image. For example, two occupants (specifically, adriver and a passenger) are riding on the vehicle VCL1 reflected in thein-vehicle face video.

In a case where the in-vehicle face video displayed on the display unit93 of the client terminal 90 is viewed, the operator OP1 may want toretrieve a number plate of the vehicle VCL1 including the face imagesFCE2 and FCE3. An operation procedure in the vehicle monitoring system 1in a case of retrieving the number plate NBPL1 of the vehicle VCL1 byusing the face images FCE2 and FCE3 will be described in detail withreference to FIG. 17 which will be described later. In a case where suchretrieval is performed, a retrieval result screen WD6 illustrated inFIG. 14 is displayed on the display unit 93 of the client terminal 90.

The retrieval result screen WD6 illustrated in FIG. 14 includes at leasta start time entry field for start date and time and an end time entryfield for end date and time, the start date-and-time information and theend date and time being used for retrieval, a face image display fieldFCC displaying face images that are input as retrieval conditions, anumber-of-occupants measurement result display field HCC displaying thenumber of occupants as a retrieval result, a vehicle details informationdisplay field VCC, and a plurality of candidate persons' face imageslist display field LST2 displaying face images of a plurality ofcandidate persons extracted in a face collation process (refer to FIG.17 ). The start time entry field and the end time entry field arerespectively the same as the start time entry field FR1 and the end timeentry field TO1 illustrated in FIG. 7 , and thus description thereofwill be omitted here.

The face image display field FCC displays a face image that is imagedata which is input as a retrieval condition. In the example illustratedin FIG. 14 , the face images FCE2 and FCE3 of the occupants of thevehicle VCL1 in the in-vehicle face video (refer to the abovedescription) are displayed.

The vehicle details information display field VCC displays variouspieces of vehicle information VCLIF1 regarding the vehicle VCL1reflected in the in-vehicle face video, including the number plate NBPL1as a retrieval result of a number plate using the face images.

A list of face images of a plurality of candidate persons which may bethe occupant of the vehicle VCL1 in the in-vehicle face video (refer tothe above description) is displayed in the candidate persons' faceimages list display field LST2. In the same manner as in FIG. 10 , in acase where a matching face image is identified from the face collationlist (refer to the database 57) in the process of collating face imagesof the occupant of the vehicle VCL1 in the in-vehicle face video witheach other (refer to FIG. 17 ), the vehicle monitoring application maydisplay the popup screen CPR1. Details of the popup screen CPR1 havebeen described with reference to FIG. 10 , and thus description thereofwill be omitted here.

A retrieval result screen WD7 illustrated in FIG. 15 is displayed by thevehicle monitoring application due to an operation of the operator OP1on the retrieval result screen WD6. The retrieval result screen WD7includes at least a vehicle basic information display field COND2including the number plate NBPL1 and the vehicle VCL1 as retrievalresults, the vehicle information VCLIF1, a number plate details displayfield DETL1, and the moving image reproduction screens MVFC1 and MVLP1.In the same manner as in FIG. 7 , an image (refer to the number plateimage LPcap1) of the vehicle VCL1 attached with the number plate NBPL1is displayed in the number plate details display field DETL1. A contentof the number plate attribute information LPIF1 is the same as thecontent described with reference to FIG. 7 , and thus descriptionthereof will be omitted here. Similarly, contents of the moving imagereproduction screens MVFC1 and MVLP1 are the same as the contentsdescribed with reference to FIG. 8 (or FIG. 9 ), and thus descriptionthereof will be omitted here.

A retrieval result screen WD8 illustrated in FIG. 16A is displayed bythe vehicle monitoring application due to an operation of the operatorOP1 on the retrieval result screen WD6. The retrieval result screen WD8displays the vehicle basic information display field COND2 including thenumber plate NBPL1 and the vehicle VCL1 as retrieval results, and avehicle list display field LST3 for images of vehicles respectivelyattached with a plurality of number plate candidates extracted in anumber plate collation process (refer to FIG. 17 ). In a case where itis detected that any one vehicle image is designated from the vehiclelist display field LST3 through an operation of the operator OP1, thevehicle monitoring application may display various pieces of vehicleinformation regarding the designated vehicle image with a balloon.

A retrieval result screen WD9 illustrated in FIG. 16B is displayed bythe vehicle monitoring application due to a designation operation of theoperator OP1 on any vehicle image in the retrieval result screen WD8.The retrieval result screen WD9 includes at least a start time entryfield for start date and time and an end time entry field for end dateand time, the start date-and-time information and the end date and timebeing used for retrieval, the face image display field FCC displayingface images that are input as retrieval conditions, thenumber-of-occupants measurement result display field HCC displaying thenumber of occupants as a retrieval result, the vehicle detailsinformation display field VCC, and the time line display fields TML1 andTML2.

The time line display field TML1 displays, in a time series, the dateand time on which a vehicle corresponding to any one vehicle imagedesignated on the retrieval result screen WD8 is detected by the vehicleimaging camera. In the example illustrated in FIG. 16B, the vehicle inthe selected vehicle image is detected between “12:20:30” and “12:20:34”by a vehicle imaging camera having the name of “LPR Camera”.

The time line display field TML2 displays, in a time series, the dateand time on which a vehicle corresponding to any one vehicle imagedesignated on the retrieval result screen WD8 is detected by the vehicleimaging camera. In the example illustrated in FIG. 16B, the vehicle inthe selected vehicle image is detected between “12:20:30” and “12:20:34”by a vehicle imaging camera having the name of “LPR Camera2”. Thevehicle monitoring application may display the same screen as theretrieval result screen WD5 in FIG. 12 on the display unit 93 due to anoperation of the operator OP1 on the retrieval result screen WD6.

Next, with reference to FIG. 17 , a description will be made of a seriesof operation procedures of retrieving a number plate by using a faceimage of an in-vehicle occupant in the vehicle monitoring system 1according to Embodiment 1. FIG. 17 is a sequence diagram illustrating anexample of a series of operation procedures of retrieving a number plateby using a face image of an in-vehicle occupant that is input as aretrieval condition. In the description of FIG. 17 , similarly, aconfiguration of a vehicle imaging camera exemplifies the configurationof the vehicle monitoring system 1 illustrated in FIG. 1A, and aconfiguration of a vehicle imaging camera exemplifies the configurationof the vehicle imaging camera 10. In the description of FIG. 17 , it isassumed that, for example, as illustrated in FIG. 5A, whenever a numberplate video or an in-vehicle face video is received, the videoaccumulation/analysis server 50 acquires a result of a number platecollation process, detection results of a vehicle type and a vehiclecolor, a result of a face collation process, and a measurement result ofthe number of occupants, based on the number plate video or thein-vehicle face video (St34, St37, St40, and St43), and accumulates theresults (St44).

In FIG. 17 , in a case where a face image (for example, the face imageFCE2) that is input through an operation of the operator OP1 is input asa retrieval condition (St91), the video accumulation/analysis server 50receives a retrieval request (including the face image FCE2) sent fromthe client terminal 90. In response to the reception of the retrievalrequest, the video accumulation/analysis server 50 (for example, theprocessor PRC1) sends a face collation request for instructing acollation process for the face image FCE2 to be executed to the facedetection/collation core 53 (St92). The face detection/collation core 53collates the face image FCE2 with the face collation list preserved inthe database 57 (St93). The face detection/collation core 53 notifiesthe video accumulation/analysis server 50 of a list of results (forexample, a score, an age, a gender, a camera ID, camera positioninformation, and an imaging time point) of the face collation process(St94). The age and the gender may be determined in a face detectionprocess.

The video accumulation/analysis server 50 retrieves (extracts)corresponding accumulated information (for example, the number plate,the vehicle type and the vehicle color, the face image of the passengerin a case where there is the passenger, and the result of the number ofoccupants) from the accumulated results of the various processesaccumulated in step St44 on the basis of the score, the camera ID, andthe imaging time point included in the processing results obtained instep St94 (St95).

The video accumulation/analysis server 50 sends the face image of thepassenger extracted in step St95 and a face collation request to theface detection/collation core 53 (St96). The face detection/collationcore 53 collates the face image of the passenger with the face collationlist preserved in the database 57 (St97). The face detection/collationcore 53 notifies the video accumulation/analysis server 50 of results(for example, a score, an age, a gender, a camera ID, camera positioninformation, and a list of imaging time points) of the face collationprocess (St98).

The video accumulation/analysis server 50 sends a number collationrequest for instructing a collation process for the number plateextracted in step St95 to be executed to the number detection/collationcore 51 (St99). The number detection/collation core 51 collates thenumber plate with the number collation list preserved in the database 57(St100). The number detection/collation core 51 notifies the videoaccumulation/analysis server 50 of results (for example, the numberplate, a camera ID, an imaging time point, and vehicle registrationinformation (refer to the above description)) of the number platecollation process (St101).

The video accumulation/analysis server 50 (for example, the processorPRC1) lists scores and accumulated results associated with the cameraIDs and the imaging time points of which notifications are sent in stepsSt94, St98, and St101 (St102). The video accumulation/analysis server 50sends a result list including the accumulated results (for example,number information, vehicle registration information, informationregarding the number of occupants, and the face image of the passenger)to the client terminal 90 as a retrieval result along with the scores(St103). The client terminal 90 displays the retrieval result screen WD6(for example, refer to FIG. 14 ) by using the retrieval result sent instep St103 (St104).

As mentioned above, the vehicle monitoring system 1 according toEmbodiment 1 includes at least one vehicle imaging camera 10, and thevideo accumulation/analysis server 50 that is communicably connected tothe client terminal 90. The vehicle imaging camera 10 can image a numberand an occupant's face of a vehicle entering an angle of view thereof,and transmits a captured video in which the number of the vehicle andthe occupant's face are reflected to the video accumulation/analysisserver 50. The video accumulation/analysis server 50 acquires ananalysis result of the number, analysis results of the type and a colorof the vehicle, an analysis result of the occupant's face, and ananalysis result of the number of occupants on the basis of the capturedvideo, and accumulates as analysis results of the captured video. Thevideo accumulation/analysis server 50 sends the analysis result of thenumber, the analysis results of the type and a color of the vehicle, theanalysis result of the occupant's face, and the analysis result of thenumber of occupants to the client terminal 90 in correlation with asnapshot of the captured video of the vehicle.

Consequently, the vehicle monitoring system 1 can identify uniquefeatures (for example, a number plate of a vehicle, a face image of anoccupant of the vehicle, a vehicle type and a vehicle color of thevehicle, and the number of occupants) of a runaway vehicle on which asuspect having caused a case or the like is riding with high efficiency.Therefore, the vehicle monitoring system 1 can assist promptidentification of a suspect on the basis of an appearance of the suspectand the features of the runaway vehicle by using the unique features(refer to the above description) of the runaway vehicle, and can thusimprove convenience of police investigation.

The video accumulation/analysis server 50 includes the database 57preserving the number collation list in which a number of aninvestigation target vehicle is registered, and collates whether or notnumber information is registered in the number collation list in a casewhere the number information that is input as a retrieval condition isreceived from the client terminal 90. The video accumulation/analysisserver 50 extracts a face image of an occupant of a vehicle, the typeand a color of the vehicle, and the number of occupants corresponding tothe number information by using a collation result of the numberinformation and an analysis result of the captured video, and displaysthe extracted results on the client terminal 90. Consequently, theclient terminal 90 can present a face image of an occupant riding on avehicle corresponding to a number plate to the operator OP1 on the basisof information regarding the number plate that is input through anoperation of the operator OP1 by using the vehicle imaging camera 10that can substantially simultaneously image the number plate and a faceof the in-vehicle occupant of the vehicle as a subject.

The video accumulation/analysis server 50 extracts a face image of eachof a driver and a passenger of a vehicle, and displays the face imageson the client terminal 90. Consequently, the client terminal 90 canpresent not only a driver (for example, a main culprit in a case or thelike) but also a passenger (for example, an accomplice in the case orthe like) on a passenger seat as occupants riding on a vehiclecorresponding to a number plate that is input as a retrieval condition,to the operator OP1, without omission, and can thus improveinvestigation efficiency.

The video accumulation/analysis server 50 preserves the face collationlist in which a face image and person information of an investigationtarget person are registered in the database 57, and displays personinformation of an occupant corresponding to a face image on the clientterminal 90 in a popup form according to a collation result indicatingthat the face image of the occupant of a vehicle corresponding to numberinformation is registered in the face collation list. Consequently, in acase where an occupant of a vehicle corresponding to a number plate thatis input as a retrieval condition is identified as a criminal such as anex-convict registered in a blacklist, the client terminal 90 canexplicitly present such a fact to the operator OP1.

The video accumulation/analysis server 50 includes the database 57preserving the face collation list in which a face image and personinformation of an investigation target person are registered, andcollates whether or not a face image is registered in the face collationlist in a case where the face image that is input as a retrievalcondition is received from the client terminal 90. The videoaccumulation/analysis server 50 extracts a number of a vehicle, the typeand a color of the vehicle, and the number of occupants corresponding toa face image by using a collation result of the face image and ananalysis result of a captured video, and displays the extracted resultson the client terminal 90. Consequently, the client terminal 90 canpresent a number plate of a vehicle on which a person corresponding to aface image is riding to the operator OP1 on the basis of the face imagethat is input through an operation of the operator OP1 by using thevehicle imaging camera 10 that can substantially simultaneously imagethe number plate and a face of the in-vehicle occupant of the vehicle asa subject.

The video accumulation/analysis server 50 collates whether or not a faceimage of a passenger of a vehicle corresponding to a face image isregistered in the face collation list, and displays a collation resulton the client terminal 90 along with a number of a vehicle, the type anda color of the vehicle, and the number of occupants. Consequently, thevideo accumulation/analysis server 50 can collect a larger number ofnumber plate candidates by also taking into consideration a case where amain culprit (for example, a driver) and an accomplice (for example, apassenger) of a case or the like perform driving by turns on the run,and can thus extract number plates of runaway vehicles of a suspectgroup (main culprits and accomplices) without omission.

The video accumulation/analysis server 50 includes the database 57preserving the number collation list in which a number of aninvestigation target vehicle is registered, and collates whether or nota detection result of a number of a vehicle is registered in the numbercollation list. In a case where the detection result of the number ofthe vehicle is registered in the number collation list, or analysisresults of the type and a color of the vehicle satisfy a predeterminedcondition (for example, in a case where there is the hit), the videoaccumulation/analysis server 50 analyzes an occupant's face and thenumber of occupants on the basis of a captured video. Consequently, theoperator OP1 can recognize that various results of processes such asanalysis on a video captured by the vehicle imaging camera 10 areaccumulated in the database 57 of the video accumulation/analysis server50 and can thus also visually check contents thereof via the displayunit 93 of the client terminal 90.

The vehicle imaging camera 10 includes a database (for example, thememory 12) preserving the number collation list in which a number of aninvestigation target vehicle is registered, and collates whether or nota detection result of a number of a vehicle is registered in the numbercollation list. The video accumulation/analysis server 50 receives anotification of a collation result from the vehicle imaging camera 10,then analyzes the type and a color of the vehicle on the basis of acaptured video, and analyzes an occupant's face and the number ofoccupants on the basis of a captured video in a case where the detectionresult of the number of the vehicle is registered in the numbercollation list, or analysis results of the type and a color of thevehicle satisfy a predetermined condition (for example, in a case wherethere is the hit). Consequently, the operator OP1 operating the clientterminal 90 can recognize that various results of processes such asanalysis on a video captured by the vehicle imaging camera 10 areaccumulated in the database 57 of the video accumulation/analysis server50 and can thus also visually check contents thereof via the displayunit 93 of the client terminal 90. Since the number plate collationprocess can be executed in the vehicle imaging camera 10 compared withthe operation procedure in FIG. 5A, various processes on a videocaptured by the vehicle imaging camera 10 can be distributed to thevehicle imaging camera 10 and the video accumulation/analysis server 50,and thus it is possible to reduce a processing load on the videoaccumulation/analysis server 50.

The vehicle imaging camera 10 includes a database (for example, thememory 12) preserving the number collation list in which a number of aninvestigation target vehicle is registered, and collates whether or nota detection result of a number of a vehicle is registered in the numbercollation list. The vehicle imaging camera 10 analyzes the type and acolor of the vehicle on the basis of a captured video, and analyzes anoccupant's face and the number of occupants on the basis of a capturedvideo in a case where the detection result of the number of the vehicleis registered in the number collation list, or analysis results of thetype and a color of the vehicle satisfy a predetermined condition (forexample, in a case where there is the hit), and transmits an analysisresult of the button, analysis results of the type and the color of thevehicle, an analysis result of the occupant's face, and an analysisresult of the number of occupants to the video accumulation/analysisserver 50. Consequently, the operator OP1 operating the client terminal90 can recognize that various results of processes such as analysis on avideo captured by the vehicle imaging camera 10 are accumulated in thedatabase 57 of the video accumulation/analysis server 50 and can thusalso visually check contents thereof via the display unit 93 of theclient terminal 90. Since various processes such as number platecollation, detection of a vehicle type and a vehicle color, facecollation, and measurement of the number of occupants can be executed inthe vehicle imaging camera 10 compared with the operation procedures inFIGS. 5A and 5B, the high performance vehicle imaging camera 10 is used,and thus it is possible to remarkably reduce a processing load on thevideo accumulation/analysis server 50.

The vehicle imaging camera 10 detects a number of a vehicle on the basisof a captured video, sends a number image obtained by cutting out thenumber of the vehicle to the video accumulation/analysis server 50,detects an occupant's face of the vehicle on the basis of the capturedvideo, and sends a face image obtained by cutting out the occupant'sface to the video accumulation/analysis server 50. The videoaccumulation/analysis server 50 includes the database 57 preserving thenumber collation list in which a number of an investigation targetvehicle is registered, and collates whether or not the number image isregistered in the number collation list and analyzes the type and acolor of the vehicle and the number of occupants on the basis of thecaptured video in response to reception of the number image, andanalyzes the occupant's face on the basis of the captured video inresponse to reception of the face image. Consequently, the operator OP1operating the client terminal 90 can recognize that various results ofprocesses such as analysis on a video captured by the vehicle imagingcamera 10 are accumulated in the database 57 of the videoaccumulation/analysis server 50 and can thus also visually checkcontents thereof via the display unit 93 of the client terminal 90. Anumber plate image and a face image that are targets of collationprocesses with the number collation list and the face collation list aregenerated by cutting out only a corresponding number plate and face fromcaptured videos, and thus it is possible to reduce loads of a numberplate collation process and a face collation process.

In the vehicle monitoring system 1 according to Embodiment 1, thevehicle imaging camera 10 performs imaging while switching a firstimaging condition including an image parameter appropriate for imaging anumber of a vehicle entering an angle of view thereof and a secondimaging condition including an image parameter appropriate for imagingan occupant's face of the vehicle. The vehicle imaging camera 10transmits a first captured video (for example, a number plate video)under the first imaging condition and a second captured video (forexample, an in-vehicle face video) under the second imaging condition,to the video accumulation/analysis server 50. The videoaccumulation/analysis server 50 arranges reproduction screens for thefirst captured video and the second captured video that arereproduceable in the client terminal 90 and displays the reproductionscreens on the client terminal 90 on the basis of the first capturedvideo and the second captured video (for example, refer to FIG. 8 or 9).

Consequently, the vehicle monitoring system 1 displays the moving imagereproduction screen MVLP1 for a number plate video and the moving imagereproduction screen MVFC1 for an in-vehicle face video, the number platevideo and the in-vehicle face video captured at the substantially sametime by the vehicle imaging camera 10, on the client terminal 90, andcan thus assisting a police officer in identifying unique features (forexample, a number plate of a vehicle, a face image of an occupant of thevehicle, a vehicle type and a vehicle color of the vehicle, and thenumber of occupants) of a runaway vehicle on which a suspect havingcaused a case or the like is riding in an early stage and visually.Therefore, the vehicle monitoring system 1 can improve convenience ofpolice investigation.

The video accumulation/analysis server 50 arranges and displaysreproduction screens for the first captured video and the secondcaptured video on which a reproduction operation and a temporary stopoperation are possible in the client terminal 90, on the client terminal90. Consequently, the operator OP1 can perform temporary stop duringreproduction of moving images of a number plate video, an in-vehicleface video, or both thereof by operating the client terminal 90, and canthus stop the moment suspected by the operator OP1 to check detailsthereof.

The video accumulation/analysis server 50 further displays a numberanalysis result including a cutout number image obtained by cutting outthe number on the basis of the first captured video, and a snapshot ofthe first captured video on the client terminal 90. Consequently, theoperator OP1 can check details of a number plate at first sight toimmediately understand information regarding a runaway vehicle of asuspect, and thus it is possible to improve investigation efficiency.

An image parameter in the first imaging condition is an exposure timeequal to or less than a first reference value (default value) or a gainvalue equal to or smaller than a second reference value. An imageparameter in the second imaging condition is an exposure time more thanthe first reference value (default value) or a gain value greater thanthe second reference value. Consequently, the vehicle imaging camera 10uses the image parameter in the first imaging condition, and can thuscapture a high resolution number plate video under the first imagingcondition appropriate for characteristics (for example, the use of ahighly reflective material) of a number plate. Similarly, the vehicleimaging camera 10 uses the image parameter in the second imagingcondition, and can thus capture a high resolution in-vehicle face videounder the second imaging condition appropriate for characteristics (forexample, an occupant's face viewed through a windshield) of thein-vehicle face video.

The vehicle imaging camera 10 includes the illuminance sensor S1 thatcan detect the brightness of the surrounding, and the nighttimeshort-wavelength lighting 16 that irradiates the surrounding with IRlight, and switches an operation mode to a daytime mode or a nighttimemode on the basis of a detection result from the illuminance sensor S1.During the nighttime mode, the vehicle imaging camera 10 applies IRlight having an intensity less than a third reference value (defaultvalue) from the nighttime short-wavelength lighting 16, and also appliesIR light having an intensity more than the third reference value(default value) from the nighttime short-wavelength lighting 16.Consequently, a face of an in-vehicle occupant (for example, a driver)is clearly reflected even though the surrounding is dark in a nighttimeperiod, and thus it is possible to efficiently assist the operator OP1in recognizing the face of the occupant in an early stage. The numberplate video in the moving image reproduction screen MVLP2 is reproducedsuch that a number plate of a vehicle is clearly reflected even thoughthe surrounding is dark in a nighttime period, and thus it is possibleto efficiently assist the operator OP1 in recognizing the number platein an early stage.

In a case where an instruction for updating the image parameter in thefirst imaging condition or the second imaging condition is received inresponse to a user's operation on the client terminal 90 on which areproduction screen for the first captured video and a reproductionscreen for the second captured video are displayed, the videoaccumulation/analysis server 50 sends the instruction for updating theimage parameter to the vehicle imaging camera 10. The vehicle imagingcamera 10 updates the corresponding image parameter in response to theinstruction for updating the image parameter. Consequently, the vehicleimaging camera 10 can generate an in-vehicle face image captured on thebasis of image parameters appropriate for imaging a face of an occupantin a vehicle and can generate a number plate image captured on the basisof image parameters appropriate for imaging a number plate by switchingthe image parameters for the in-vehicle face image and the imageparameters for the number plate image every frame.

The video accumulation/analysis server 50 causes the client terminal 90to reproduce the first captured video and the second captured videogenerated by the vehicle imaging camera 10 at the substantially sametime (including the same time). Consequently, an in-vehicle face videoin the moving image reproduction screen MVFC1 is reproduced such that aface of an in-vehicle occupant (for example, a driver) is clearlyreflected regardless of the daytime or the nighttime, and thus it ispossible to efficiently assist the operator OP1 in recognizing the faceof the occupant in an early stage. On the other hand, the number platevideo in the moving image reproduction screen MVLP1 is reproduced suchthat a number plate of a vehicle is clearly reflected regardless of thedaytime or the nighttime, and thus it is possible to efficiently assistthe operator OP1 in recognizing the number plate in an early stage.

As mentioned above, various embodiments have been described withreference to the drawings, but, needless to say, the present disclosureis not limited to the embodiments. It is obvious for a person skilled inthe art to conceive of various modifications, alterations, replacements,additions, deletions, and equivalents within the category disclosed inthe claims, and it is understood that they are also naturally includedin the technical scope of the present disclosure. The respectiveconstituent elements in the various embodiments may be optionallycombined with each other within the scope without departing from thespirit of the invention.

In Embodiment 1, a vehicle has been exemplified as a detection target ina video captured by the vehicle imaging camera 10, but a detectiontarget is not limited to a vehicle, and may be other objects (forexample, a moving object such as a vehicle). Other objects may be aflying object such as a drone operated by a person such as a suspecthaving caused a case or the like.

The present disclosure is useful as a vehicle monitoring system and avehicle monitoring method capable of assisting a police officer inidentifying unique features of a runaway vehicle on which a suspecthaving caused a case or the like is riding in an early stage andvisually, and thus improving convenience of police investigation.

The present application is based upon Japanese Patent Application(Patent Application No. 2019-094824 filed on May 20, 2019), the contentof which is incorporated herein by reference.

What is claimed is:
 1. A vehicle monitoring system comprising: a serverthat is communicably connected to at least one camera; and a clientterminal that is communicably connected to the server, wherein theserver stores a collation list used for collating a license plate,receives a first video data from the camera, the first video data beingcaptured under a first capturing condition including a first parameterfor capturing a license plate provided on a vehicle entering an angle ofview of the camera, and acquires a license plate collation result of thevehicle based on the collation list and the first video data, andwherein the client terminal sends, to the server, a search condition ofa license plate based on a user operation, displays, on a display, alist of license plate search results which satisfy the search condition,in response to selection of at least one of the license plate searchresults based on the user operation, displays, on the display, detailinformation of a selected license plate corresponding to the at leastone of the license plate search results, and displays, on the display, alicense plate image in which an entire of a vehicle corresponding to theselected license plate is appearing as the detail information of theselected license plate.
 2. The vehicle monitoring system according toclaim 1, wherein the client terminal further displays an attributeinformation including a cut-off image obtained by cutting off theselected license plate in the license plate image.
 3. The vehiclemonitoring system according to claim 2, wherein the attributeinformation further includes a text data corresponding to a readoutresult of the selected license plate, a country name, a color of theselected license plate and a score.
 4. The vehicle monitoring systemaccording to claim 1, wherein the client terminal displays a movieviewer icon of the vehicle corresponding to the selected license plateas the detail information of the selected license plate.
 5. The vehiclemonitoring system according to claim 4, wherein the server receives asecond video data from the camera, the second video data being capturedunder a second capturing condition including a second parameter forcapturing a face of an occupant riding in a vehicle entering the angleof the camera, and wherein the client terminal in response to selectionof the movie viewer icon based on the user operation, displays areproduction screen of the first video data and a reproduction screen ofthe second video data, the first video data and the second video data inwhich the vehicle corresponding to the selected license plate capturedat substantially same time by the camera appears.
 6. The vehiclemonitoring system according to claim 5, wherein the client terminaldisplays the reproduction screen of the first video data and thereproduction screen of the second video data in a row.
 7. A vehiclemonitoring method comprising: acquiring a collation list used forcollating a license plate; receiving a first video data from the camera,the first video data being captured under a first capturing conditionincluding a first parameter for capturing a license plate provided on avehicle entering an angle of view of at least one camera; acquiring alicense plate collation result of the vehicle based on the collationlist and the first video data; displaying, on a display, a list oflicense plate search results which satisfy a search condition of alicense plate based on a user operation; in response to selection of atleast one of the license plate search results based on the useroperation, displaying, on the display, detail information of a selectedlicense plate corresponding to the at least one of the license platesearch results; and displaying, on the display, a license plate image inwhich an entire of a vehicle corresponding to the selected license plateis appearing as the detail information of the selected license plate.